From nobody Sun Mar 5 03:05:10 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PTmm812S9z3vsg8 for ; Sun, 5 Mar 2023 03:05:12 +0000 (UTC) (envelope-from leres@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PTmm802zjz3Mxf for ; Sun, 5 Mar 2023 03:05:12 +0000 (UTC) (envelope-from leres@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1677985512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cJ4rBxPbZ6RILaRsX8txAX65s0wdAom0Vd21n59wcJA=; b=RLma3s12zOJ50tn0E32n6g4KtT9gMevkua9nJ6QTWmrUiBosfw0/nl+JGI+JUN8Xn0toGV 4DegERxU9D/7+eIjTbKtY7UkfNRLyqrtIjyi8qUeT1WVQ+fKiPFCsbHPFYvTRCgg0NJyqb XX6BlHkJpM6u0Clo7WCHo1QNG2KXi+SHGnvdONBfEkChYvAICgvJTKcfWBh6UJqkAEjvMl QPGWhgG+PoDNiQtdm0gFleF8HTsYB9Io1R1afuPTJtaUMzxDmqMhWkN7CRUBIJI5Jt6x3b mDbRv5fUsBBAuD6x4FvZNKnhYLcSZl8Y42y8OHmroOurJlw8kdgN2fg7uT7QsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1677985512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cJ4rBxPbZ6RILaRsX8txAX65s0wdAom0Vd21n59wcJA=; b=hkkYWf8eT9smHCnEBHxqopYz0PiaqHVL7mbS2yN+DwxGf2wNUp1UJVqqUlx99RHw4NBDfg s+XkCSM/iyMDT7VMvitxWkEjXbuNp/aufveHXa84t7UXsWdPW+NW6fo+yBOFzpoerQboXM ldWl39899pP86H0z19vGyk6oQvcmLHtaGu+ZpC/in15TTQr0/mI3ebVAGqRJGcyjbsjiPm kdvE23HUMGRu4lmwCnPt1afqE5hXif07NkodRWeIUfqBSMT2TSkc+sQBuqZS6xJR0ko9jx E0kzV+Aed/abjsV8cU2E4T6vZSVhEDdq9aw2We/ZaZ/uxWTD16TUXRMEjb20+g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1677985512; a=rsa-sha256; cv=none; b=rJ6Zem/5l3wxHk5SM4fyxhV/Qj6XRFZBWiixrz2UeVV326fYYp2NP9WKtt4rLzg0FBePoa xcezooktvcM9kjMAt4vDIOqoxPOJSCNPkD1GrOhIbnzA8OmNc0aZ0HZRqd5Y10EPh2umPW z6Xs69CGTUiWpMpeTrtfmgmnqSZaNBC/YwmRx9OMXrVORCzq5LFHerZSLxziUk8E23f4Ds UJSe2ZjUw/u1roVBN9edURGEoyF33tsFltPafS3q6UjalMPSa0pTS1Qn7deYCzpaw4dTgR 0dv7BoOsm9+Hi4tTdYJFyu/mbo4BVVkyanwiCaI1GP1SZOmtPHDXu7jz4N+mTQ== Received: from [IPV6:fd:1965::2] (unknown [IPv6:2600:1700:ab1b:6800::49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: leres) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PTmm753mCzRNy for ; Sun, 5 Mar 2023 03:05:11 +0000 (UTC) (envelope-from leres@freebsd.org) Message-ID: <20589e6e-dbb2-7f0b-9f08-3870fdfe18f1@freebsd.org> Date: Sat, 4 Mar 2023 19:05:10 -0800 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 From: Craig Leres Subject: Re: FreeBSD+samba as a time machine server for OSX/Ventura? To: freebsd-hackers@freebsd.org References: <7cda0c71-5fe2-722b-4ff9-0b48807b7627@langille.org> Content-Language: en-US In-Reply-To: <7cda0c71-5fe2-722b-4ff9-0b48807b7627@langille.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N I finally got a chance to work on this today. The many followups are appreciated. samba413 -> samba416 is definitely something I was missing. I think there was only 4.1.3 when I first switched to samba. Not sure why netatalk3 stopped working for me. Nice to see Dan's proof that this is still a viable time machine solution. I think I'll stick with samba as it may one day allow me to backup my windows laptop. I am using zfs on the FreeBSD side. The fruit:zero_file_id situation is interesting but I'm on 13.1-RELEASE so I don't think it applies to me yet. But nice that I won't run into it once I do upgrade to 13.2. I have not previously run any kind of mdns on the FreeBSD system; I tried my hand at a minimal avahi config that I think works. But I think this just makes it possible to browse for the remote server from the time machine settings page. I had previously just been using %K in finder to connect to server. (Maybe this will help me with windows down the road.) I found a different recipe to get at the logs: log show --predicate 'subsystem == "com.apple.TimeMachine"' --info --last 6h | grep -F 'eMac' | grep -Fv 'etat' | awk -F']' '{print substr($0,1,19), $NF}' Something that popped out right away is that the mac was trying to get into a uuid named directory in 'file:///Volumes/.timemachine/XYZ._smb._tcp.local.' that did not exist. Then I found many "Backup of " directories in /Volumes. This all made me think that the mac was trying to continue using the pre-ventura backup so I nuked the files that were in that directory (and the failed files on the server) and rebooted the mac. After re-adding the remote volume, I have a time machine backup in progress for the first time since 2021. Thanks to all; freebsd-hackers (as usual) for the win! Craig From nobody Sun Mar 5 16:47:58 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PV71Z1Hgbz3wgVD for ; Sun, 5 Mar 2023 16:48:02 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PV71X6wCyz3H42; Sun, 5 Mar 2023 16:48:00 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=Zd18tv07; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::22f as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oi1-x22f.google.com with SMTP id r40so5433214oiw.0; Sun, 05 Mar 2023 08:48:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678034879; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=YcI1MWUg9+/lqKHBvtQ1HUNRLTCOvIlc4YJ69TsjxmA=; b=Zd18tv07Xhlf7PP26dkF3jLjoE6vov2kYhptegrM19XA/TWyI0uIc/v5ojU6LOxvMG /pfwdLZNu2hDXjvZI5doi/vh6W3qGOskR6ju2ndrIKFRQQ+b8GHK2MVcWKL+LyxCyrvj FpQzAWYWG5FeGM3W24AoJ++/ezvMC8l0i8GA8yqGlfjkJDjA/mOPr1JiBnS8fnfWrUpP XEl1jswCyluQrBCn1c1HjaOgVI2nBEoaNaZlc5maEU0FSApq1tevK6+HDXRCW0zMHxgI 86K5clNG0xgtP0U+LvHUKmyaEN6Vt8eWOZyodyo/tYbnrXLdPdXfZfQ9QcsYne7rGHBj iUUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678034879; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YcI1MWUg9+/lqKHBvtQ1HUNRLTCOvIlc4YJ69TsjxmA=; b=pn3lDkeA929u/vnhDtqtlP/hha/zsbxfaB9/BP/yxZMCCS/dpQU5AQ8XMvHEwZgG7v X4U1ZqS1sG35xLR9R9uwb/txf7sNi9T0cM5ECZj2JgnFSgKjWay+4ufyB4YdRQER1q3E 1N8NetkGVQfjo2N3Oipkqiy71M6rBJU9xwIYRQXcyQGR/zz2Cqd9KufhVM2MtTfNGOAJ TeAS/aTcS5mYTiwKtXQCp5DrJJY6K5mstlOGsrCN4jvUb8OO8cSWXjm5GIer09G1Nu5T mw+5sXe8KeZEJYi7aKgIIfUYK6xCiv8qd6/7629j4AQW2dmDSoa1O3H7kTt9DwX5gWi4 bQEQ== X-Gm-Message-State: AO0yUKXrejOUn5Blg6lyHl3lqqtNcsovQJ8/oT3Bd83enlvNgcfDaNpt v9yVFxI2b1qVzsVAUxuJ8jxKtQaxxamkFc7QvJBSl79H X-Google-Smtp-Source: AK7set94LGt/h9fNjTIzV/KXRDCwmocUAn35pSIpN2DqAfEzWS/HHd04/R9ZdLoeumhC+ENdx/elyjSwfPvfGQj1l0M= X-Received: by 2002:a05:6808:30c:b0:383:e7b5:8177 with SMTP id i12-20020a056808030c00b00383e7b58177mr2523609oie.11.1678034879504; Sun, 05 Mar 2023 08:47:59 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a05:6802:31f:b0:4c2:d201:fe1f with HTTP; Sun, 5 Mar 2023 08:47:58 -0800 (PST) In-Reply-To: <2836EF0C-8B4B-441C-927B-3796457A3865@FreeBSD.org> References: <2f3dcda0-5135-290a-2dff-683b2e9fe271@FreeBSD.org> <2836EF0C-8B4B-441C-927B-3796457A3865@FreeBSD.org> From: Mateusz Guzik Date: Sun, 5 Mar 2023 17:47:58 +0100 Message-ID: Subject: Re: CFT: snmalloc as libc malloc To: David Chisnall Cc: freebsd-hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.79 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.79)[-0.790]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22f:from]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_FROM(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4PV71X6wCyz3H42 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Apologies for significant delay, got other stuff and this fell through the cracks. I'm going to start with snmalloc remarks, memcpy-related response is down b= elow. I find it quite weird there are absolutely no mentions of NUMA in a general-purpose allocator made in recent years. While nothing can be done for an application which does not use any affinity, what *can* be done is not negatively affecting programs which *do* use it. Most notably making sure that allocations backed by one page don't land in threads from distinct domains. It is unclear to me what, if anything, is the allocator doing on this front, but I also don't know if this would be a regression vs jemalloc. One major issue I found is the use of hand-rolled spinlocks -- while they may be fine for specialized uses, they are a non-starter for general purpose. This is the age-old problem of priority problems where you can end up yield()ing all you want, while the lock owner nowhere near CPU or the scheduler does not think you should give way to it. The least bad thing to do would be to add umtx support to remedy this problem, but that's a lot of work and it is probably not warranted. Then the fallback is pthread locks -- while pessimal, they don't suffer the problem. The lock operation not being in the fast path is not a factor for this problem. I tried to do a sensible benchmark of the thing, but ran into snags to sort out first. The idea was to 'tinderbox' it, except: 1. make's use of poll is a massive scalability bottleneck, addressed here in an experimental manner: https://github.com/macdice/freebsd/tree/bmake-shmem 2. with that out of the way there is still massive contention in the vm subsystem. I had a patch for most of it here: https://reviews.freebsd.org/D36038 . it rebases cleanly but no longer works, I don't know why yet There is other issues. Ultimately if snmalloc is to ship, it would be nice ot have it for 14 and we are nearing the time where it will be too late, so I'll try to sort it all out next week. Meanwhile there is the lock issue to sort out. On to memcpy On 2/9/23, David Chisnall wrote: > On 9 Feb 2023, at 20:53, Mateusz Guzik wrote: >> >> First and foremost, perhaps I should clear up that I have no opinion >> on snmalloc or it replacing jemalloc. I'm here only about the memcpy >> thing. >> >> Inspecting assembly is what I intended to do, but got the compilation >> problem. > > If you just want to look at the memcpy, you might do better using the > upstream (CMake) build system, which builds a shim library that you can > disassemble and look at. > >> So, as someone who worked on memcpy previously, I note the variant >> currently implemented in libc is pessimal for sizes > 16 bytes because >> it does not use SIMD. I do have plans to rectify that, but ENOTIME. > > That=E2=80=99s one of the benefits of the C++ implementation. We were ab= le to try a > few dozen variations in a morning. Writing a single one of those in > assembly would take a similar amount of time. > > We were inspired by the automemcpy paper, which demonstrated that you can > write memcpy implementations tuned to specific workloads in higher-level > languages and get better performance. > >> I also note CPUs are incredibly picky when it comes to starting point >> of the routine. The officialy recommended alignment of 16 bytes is >> basically a tradeoff between total binary size and performance. Should >> you move the routine at different 16 bytes intervals you will easily >> find big variation in performance, depending on how big of an >> alignment you ended up with. > > Yes, that=E2=80=99s what our test does. It does a large number of differ= ent copies > with different sizes and alignments. > I'm not talking about placement of buffers passed to memcpy, I'm talking about placement of *code implementing memcpy*. >> I'm trying to say that if you end up with different funcs depending on >> bounds checking and you only align them to 16 bytes, the benchmark is >> most likely inaccurate if only for this reason. > > I=E2=80=99m not sure I follow this, sorry. > If the address of the *func itself* is divisible by 16 but not 32, it may end up with a different performance profile depending on uarch. And the code may randomly move up and down as you change the source, thus stabilizing it at some value (say 32) would be the best thing to do. >>> The fastest on x86 is roughly: >>> >>> - A jump table of power for small sizes that do power-of-two-sized smal= l >>> copies (double-word, word, half-word, and byte) to perform the copy. >> >> Last I checked this was not true. The last slow thing to do was to >> branch on few sizes and handle them with overlapping stores. Roughly >> what memcpy in libc is doing now. >> >> Jump table aside, you got all sizes "spelled out", that is just >> avoidable impact on icache. While overlapping stores do come with some >> penalty, it is cheaper than the above combined. > > They=E2=80=99re not all spelled out, because a lot of them are subsets of= the > others. The compiler does a *very* good job of optimising this. The > generated assembly was a lot more complex than anything I could write by > hand. > I said they are spelled out in the code which ends up being generated, it looks like this: [snip] mov (%rsi),%rax mov %rax,(%rdi) pop %rbp ret mov (%rsi),%rax mov %rax,(%rdi) movzbl 0x8(%rsi),%eax mov %al,0x8(%rdi) pop %rbp ret mov (%rsi),%rax mov %rax,(%rdi) movzwl 0x8(%rsi),%eax mov %ax,0x8(%rdi) pop %rbp ret [/snip] That section is *massive* and it is pessimal, once more the thing to do is to do a few branches to land in sections handling stuff with overlapping stores, akin to what bionic is doing here: https://android.googlesource.com/platform/bionic/+/refs/heads/master/libc/a= rch-x86_64/string/sse2-memmove-slm.S >> I don't have numbers nor bench code handy, but if you insist on >> contesting the above I'll be glad to provide them. >> >>> - A vectorised copy for medium-sized copies using a loop of SSE copies. >> >> Depends on what you mean by medium and which simd instructions, but >> yes, they should be used here. > > This could probably do with more tuning, but all of this is configurable > with a couple of template parameters. If it=E2=80=99s useful to have mor= e variants > for different microarchitectures then it=E2=80=99s a trivial tweak to gen= erate > them. > >>> - rep movsb for large copies. >> >> There are still old cpus here and there which don't support ERMS. They >> are very negatively impacted the above and should roll with rep stosq >> instead. > > We tested on some microarchitectures without FRMS but didn=E2=80=99t comp= are with > rep stosq as an alternative. The rep movsb is inline assembly > (https://github.com/microsoft/snmalloc/blob/4370a23f3e5f1d1d06967b1e0d616= 2bf1ee81196/src/snmalloc/global/memcpy.h#L351) > so it would be quite easy to compare. Very happy to take patches that ma= ke > things faster! > > Each tuned version is a dozen lines of code (plus a lot of comments to > explain *why* it does things a particular way), so it=E2=80=99s very easy= to add > different versions with different tuning. > >> I see the code takes some care to align the target buffer. That's >> good, but not necessary on CPUs with FSRM. > > It doesn=E2=80=99t hurt though, on the sizes where it=E2=80=99s actually = beneficial to use > the rep sequence the cost of alignment is negligible. > >> All that said, I have hard time believing the impact of bounds >> checking on short copies is around 20% or so. The code to do it looks >> super slow (not that I know to do better). > > The last time I looked at the disassembly, the code for the bounds check > touched three cache lines and has two branches. It fits well in a > superscalar pipeline so adds a handful of cycles. This is basically in t= he > noise for large copies but is double-digit percentage overhead for small > ones. > I'll be blunt here: that's misleading at best and intellectually dishonest at worst. By your own admission very small copies are most common and the very paper you refer to (automemcpy) gives a figure of 96% real-world copies being 128 bytes of less. These are also the copies which are most affected. As I tried to explain super short copies have very little to do, and the impact stated above can't be an accurate estimation on top of an ~optimal routine. Your benchmark code artificially pessimizes comparison to whatever is in libc because calls go through PLT, compared to calls to your own routine which don't. The difference does matter for short copies. I ran a quick check against glibc as shipped with Ubuntu 22. Your routine was massively slower for sizes < 12 and then started evening up with glibc, after that it was going back and forth due to what's likely just measurement error. > My original version extended the FreeBSD assembly memcpy with a call to a > function that did the bounds checks, but that had *much* worse performanc= e. > We could insert assembly to do the bounds checks, but that=E2=80=99s hard= to update > if we change the metadata layout in malloc. > I'm not saying the variant as shipped now is any good or should be used, I'm saying overlapping stores need to be used as opposed to a jump table. The current code should be whacked. Even then of course it had to get slower -- you added a func call in there. > We could modify the ifunc resolvers to use the current memcpy if you don= =E2=80=99t > care about bounds checks. I did a tiny amount of testing (mostly clang a= nd > lld) in this configuration and it was slower than using the snmalloc memc= py, > but I didn=E2=80=99t do enough benchmarking to be confident of that. Hen= ce the > CFT. > Once more see previous remarks of the func currently implemented not being = fast. The thing *to do* in the area is to implement sensible memcpy, it may be the automemcpy paper would be good basis to do it. --=20 Mateusz Guzik From nobody Sun Mar 5 17:27:16 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PV7tt6SQ6z3wjL3 for ; Sun, 5 Mar 2023 17:27:18 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PV7tt5xz9z3MJy; Sun, 5 Mar 2023 17:27:18 +0000 (UTC) (envelope-from theraven@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678037238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cK/GyFeuV4ID2PebPxUKpVhBQzWglWiusR35D02TV74=; b=MsAPhZrNLeIiS3dAL0+NQt0kUVGYWy0JE2d5fEx7qWfLcRidz8MC3BIgCjhteb2rPeOx7m oEJqH4HshGJ0tBdrdAH1e77C6I6k6sS3XaLy2RvygwqfSVzRRnVsebu93lmlE0galmEkLi At+gEHASjlMEhAxDJ6mrP2cBkczYALxYQkaPb5CBkNKuzTRXLBDaISbPkMWdXssgqcSHRe xS67sW7DvKV/0aDsLK1428ZUHr9Zq69f66p4ZpUPprD99oj0/dcEsYK8johEo0ON4UP9+Y uwB+FsB7o1VjqB/JtKfOpXcS9b/jAzFzwiGv/mTRdWnf/mXJezbaekSjS5d0jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678037238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cK/GyFeuV4ID2PebPxUKpVhBQzWglWiusR35D02TV74=; b=IucZsB2nXwS97+pfsqlQ42vsRzYCmfwDztJHP+JulvHtI5b++yhAJrUrT3jxeIqCuVYEgz qg3KIX0lL25B2Bih7FOiNqjNDNqKmGL4AMZ36lEotkLQ5eouLnoiIs5uE7r0prgR/JCQe8 Wc47SVFlVYGj11mXR+iHtXFu2RSZiKuDUcTiFrfSs/keS3gAIT6saYzX7HYpINGCTh1d6B cYvgGyzlFhK7SaOWkQ/UFxuoNwcJndq4cVmk0kAUkVPa+jYjzK5rwLMrEz2l1LKbdXv7P7 Y5/YTvL/jAP8jnc9Kg1JnVDu+nirQNGy/c6VGo3ZJE+lstjDCykGwRRhE5/JPQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678037238; a=rsa-sha256; cv=none; b=kt/PLdrvtC6QJPh5gAN0RrSm7z7QkkdylilDOhWYxelJLnkOwBrB90QTyh5yaLr8MUXtPB W/tuw+dbVaHANCobGlAlnfS1/35J4u4PnxRASmC+CyWQ9MCIO8k46XjrnxFjuA/mfwfF/C KMKQp3nvoJEAwo3gPLje8kCblWjbJFSnVZ6ClRZlvL+mN9Dv4zJXtbVdfkTv5USO8NzDiD tA3EpL2GR//8fRlu+y8SKEiN9rGqbe5mCKdM83PnFjcvVn/RZ4v+6gaFnEtlgB5iVqMCZy 1ipcX/kudPm/rMUsY5MVgz4MLzg3vw+E7xTnQn6eJabxAgMQaGWB0FAzlVwywg== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PV7tt4pfvzhYJ; Sun, 5 Mar 2023 17:27:18 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtpclient.apple (host86-166-82-133.range86-166.btcentralplus.com [86.166.82.133]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 8D3AE108DC; Sun, 5 Mar 2023 17:27:17 +0000 (GMT) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: CFT: snmalloc as libc malloc From: David Chisnall In-Reply-To: Date: Sun, 5 Mar 2023 17:27:16 +0000 Cc: freebsd-hackers Content-Transfer-Encoding: quoted-printable Message-Id: References: <2f3dcda0-5135-290a-2dff-683b2e9fe271@FreeBSD.org> <2836EF0C-8B4B-441C-927B-3796457A3865@FreeBSD.org> To: Mateusz Guzik X-Mailer: Apple Mail (2.3654.120.0.1.13) X-ThisMailContainsUnwantedMimeParts: N Hi, Thanks for the comments. =20 The FlagLock is used in two places: - Early init, before anything in libc can be guaranteed to exist. - Some *very* slow paths, where contention is incredibly improbable. We could replace them with _umtx_op / futex locks, but I doubt you=E2=80=99= d see a difference. Do you have any workloads where locks introduce = contention? The locks are all abstracted away in a single place, so it = would be fairly simple to add a couple of lock functions to the PAL. = Probably a 10-line change in total, but since it hasn=E2=80=99t shown up = in any benchmarking that we=E2=80=99ve done yet it hasn=E2=80=99t been a = priority. With respect to NUMA, if a thread is pinned to a core and the kernel is = handling it local memory, then you should see good performance = initially. All allocations are satisfied from a pool that is local to = the core. If they are freed in other threads then they are batched and = returned to the allocating thread. It is possible for chunks that are = originally created on one core to be recycled and used on another core. = This will happen at >page granularity and so is better handled in the = kernel. If you have a benchmark for memcpy that shows a big slowdown with the = bounds checking, I would like to see it so that we can use it to = optimise the implementation. 12-byte memcpys are common, but also fit = well in the store queues overlapped with the bounds checks. I=E2=80=99d = be *very* interested in workloads where this is not the case. I broke my shoulder a week ago and so am typing very slowly with one = hand, but patches are very welcome. There is a PR under review at the = moment to add memmove, which could benefit from some more tuning before = it=E2=80=99s merged. David > On 5 Mar 2023, at 16:47, Mateusz Guzik wrote: >=20 > Apologies for significant delay, got other stuff and this fell through > the cracks. >=20 > I'm going to start with snmalloc remarks, memcpy-related response is = down below. >=20 > I find it quite weird there are absolutely no mentions of NUMA in a > general-purpose allocator made in recent years. While nothing can be > done for an application which does not use any affinity, what *can* be > done is not negatively affecting programs which *do* use it. Most > notably making sure that allocations backed by one page don't land in > threads from distinct domains. It is unclear to me what, if anything, > is the allocator doing on this front, but I also don't know if this > would be a regression vs jemalloc. >=20 > One major issue I found is the use of hand-rolled spinlocks -- while > they may be fine for specialized uses, they are a non-starter for > general purpose. This is the age-old problem of priority problems > where you can end up yield()ing all you want, while the lock owner > nowhere near CPU or the scheduler does not think you should give way > to it. The least bad thing to do would be to add umtx support to > remedy this problem, but that's a lot of work and it is probably not > warranted. Then the fallback is pthread locks -- while pessimal, they > don't suffer the problem. The lock operation not being in the fast > path is not a factor for this problem. >=20 > I tried to do a sensible benchmark of the thing, but ran into snags to > sort out first. The idea was to 'tinderbox' it, except: > 1. make's use of poll is a massive scalability bottleneck, addressed > here in an experimental manner: > https://github.com/macdice/freebsd/tree/bmake-shmem > 2. with that out of the way there is still massive contention in the > vm subsystem. I had a patch for most of it here: > https://reviews.freebsd.org/D36038 . it rebases cleanly but no longer > works, I don't know why yet >=20 > There is other issues. >=20 > Ultimately if snmalloc is to ship, it would be nice ot have it for 14 > and we are nearing the time where it will be too late, so I'll try to > sort it all out next week. >=20 > Meanwhile there is the lock issue to sort out. >=20 > On to memcpy >=20 > On 2/9/23, David Chisnall wrote: >> On 9 Feb 2023, at 20:53, Mateusz Guzik wrote: >>>=20 >>> First and foremost, perhaps I should clear up that I have no opinion >>> on snmalloc or it replacing jemalloc. I'm here only about the memcpy >>> thing. >>>=20 >>> Inspecting assembly is what I intended to do, but got the = compilation >>> problem. >>=20 >> If you just want to look at the memcpy, you might do better using the >> upstream (CMake) build system, which builds a shim library that you = can >> disassemble and look at. >>=20 >>> So, as someone who worked on memcpy previously, I note the variant >>> currently implemented in libc is pessimal for sizes > 16 bytes = because >>> it does not use SIMD. I do have plans to rectify that, but ENOTIME. >>=20 >> That=E2=80=99s one of the benefits of the C++ implementation. We = were able to try a >> few dozen variations in a morning. Writing a single one of those in >> assembly would take a similar amount of time. >>=20 >> We were inspired by the automemcpy paper, which demonstrated that you = can >> write memcpy implementations tuned to specific workloads in = higher-level >> languages and get better performance. >>=20 >>> I also note CPUs are incredibly picky when it comes to starting = point >>> of the routine. The officialy recommended alignment of 16 bytes is >>> basically a tradeoff between total binary size and performance. = Should >>> you move the routine at different 16 bytes intervals you will easily >>> find big variation in performance, depending on how big of an >>> alignment you ended up with. >>=20 >> Yes, that=E2=80=99s what our test does. It does a large number of = different copies >> with different sizes and alignments. >>=20 >=20 > I'm not talking about placement of buffers passed to memcpy, I'm > talking about placement of *code implementing memcpy*. >=20 >>> I'm trying to say that if you end up with different funcs depending = on >>> bounds checking and you only align them to 16 bytes, the benchmark = is >>> most likely inaccurate if only for this reason. >>=20 >> I=E2=80=99m not sure I follow this, sorry. >>=20 >=20 > If the address of the *func itself* is divisible by 16 but not 32, it > may end up with a different performance profile depending on uarch. > And the code may randomly move up and down as you change the source, > thus stabilizing it at some value (say 32) would be the best thing to > do. >=20 >>>> The fastest on x86 is roughly: >>>>=20 >>>> - A jump table of power for small sizes that do power-of-two-sized = small >>>> copies (double-word, word, half-word, and byte) to perform the = copy. >>>=20 >>> Last I checked this was not true. The last slow thing to do was to >>> branch on few sizes and handle them with overlapping stores. Roughly >>> what memcpy in libc is doing now. >>>=20 >>> Jump table aside, you got all sizes "spelled out", that is just >>> avoidable impact on icache. While overlapping stores do come with = some >>> penalty, it is cheaper than the above combined. >>=20 >> They=E2=80=99re not all spelled out, because a lot of them are = subsets of the >> others. The compiler does a *very* good job of optimising this. The >> generated assembly was a lot more complex than anything I could write = by >> hand. >>=20 >=20 > I said they are spelled out in the code which ends up being generated, > it looks like this: > [snip] > mov (%rsi),%rax > mov %rax,(%rdi) > pop %rbp > ret > mov (%rsi),%rax > mov %rax,(%rdi) > movzbl 0x8(%rsi),%eax > mov %al,0x8(%rdi) > pop %rbp > ret > mov (%rsi),%rax > mov %rax,(%rdi) > movzwl 0x8(%rsi),%eax > mov %ax,0x8(%rdi) > pop %rbp > ret > [/snip] >=20 > That section is *massive* and it is pessimal, once more the thing to > do is to do a few branches to land in sections handling stuff with > overlapping stores, akin to what bionic is doing here: > = https://android.googlesource.com/platform/bionic/+/refs/heads/master/libc/= arch-x86_64/string/sse2-memmove-slm.S >=20 >>> I don't have numbers nor bench code handy, but if you insist on >>> contesting the above I'll be glad to provide them. >>>=20 >>>> - A vectorised copy for medium-sized copies using a loop of SSE = copies. >>>=20 >>> Depends on what you mean by medium and which simd instructions, but >>> yes, they should be used here. >>=20 >> This could probably do with more tuning, but all of this is = configurable >> with a couple of template parameters. If it=E2=80=99s useful to have = more variants >> for different microarchitectures then it=E2=80=99s a trivial tweak to = generate >> them. >>=20 >>>> - rep movsb for large copies. >>>=20 >>> There are still old cpus here and there which don't support ERMS. = They >>> are very negatively impacted the above and should roll with rep = stosq >>> instead. >>=20 >> We tested on some microarchitectures without FRMS but didn=E2=80=99t = compare with >> rep stosq as an alternative. The rep movsb is inline assembly >> = (https://github.com/microsoft/snmalloc/blob/4370a23f3e5f1d1d06967b1e0d6162= bf1ee81196/src/snmalloc/global/memcpy.h#L351) >> so it would be quite easy to compare. Very happy to take patches = that make >> things faster! >>=20 >=20 >> Each tuned version is a dozen lines of code (plus a lot of comments = to >> explain *why* it does things a particular way), so it=E2=80=99s very = easy to add >> different versions with different tuning. >>=20 >>> I see the code takes some care to align the target buffer. That's >>> good, but not necessary on CPUs with FSRM. >>=20 >> It doesn=E2=80=99t hurt though, on the sizes where it=E2=80=99s = actually beneficial to use >> the rep sequence the cost of alignment is negligible. >>=20 >>> All that said, I have hard time believing the impact of bounds >>> checking on short copies is around 20% or so. The code to do it = looks >>> super slow (not that I know to do better). >>=20 >> The last time I looked at the disassembly, the code for the bounds = check >> touched three cache lines and has two branches. It fits well in a >> superscalar pipeline so adds a handful of cycles. This is basically = in the >> noise for large copies but is double-digit percentage overhead for = small >> ones. >>=20 >=20 > I'll be blunt here: that's misleading at best and intellectually > dishonest at worst. >=20 > By your own admission very small copies are most common and the very > paper you refer to (automemcpy) gives a figure of 96% real-world > copies being 128 bytes of less. These are also the copies which are > most affected. >=20 > As I tried to explain super short copies have very little to do, and > the impact stated above can't be an accurate estimation on top of an > ~optimal routine. >=20 > Your benchmark code artificially pessimizes comparison to whatever is > in libc because calls go through PLT, compared to calls to your own > routine which don't. The difference does matter for short copies. >=20 > I ran a quick check against glibc as shipped with Ubuntu 22. Your > routine was massively slower for sizes < 12 and then started evening > up with glibc, after that it was going back and forth due to what's > likely just measurement error. >=20 >> My original version extended the FreeBSD assembly memcpy with a call = to a >> function that did the bounds checks, but that had *much* worse = performance. >> We could insert assembly to do the bounds checks, but that=E2=80=99s = hard to update >> if we change the metadata layout in malloc. >>=20 >=20 > I'm not saying the variant as shipped now is any good or should be > used, I'm saying overlapping stores need to be used as opposed to a > jump table. The current code should be whacked. >=20 > Even then of course it had to get slower -- you added a func call in = there. >=20 >> We could modify the ifunc resolvers to use the current memcpy if you = don=E2=80=99t >> care about bounds checks. I did a tiny amount of testing (mostly = clang and >> lld) in this configuration and it was slower than using the snmalloc = memcpy, >> but I didn=E2=80=99t do enough benchmarking to be confident of that. = Hence the >> CFT. >>=20 >=20 > Once more see previous remarks of the func currently implemented not = being fast. >=20 > The thing *to do* in the area is to implement sensible memcpy, it may > be the automemcpy paper would be good basis to do it. >=20 > --=20 > Mateusz Guzik From nobody Sun Mar 5 17:52:17 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PV8S44Vt4z3wkhl for ; Sun, 5 Mar 2023 17:52:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PV8S36xx1z3RDX; Sun, 5 Mar 2023 17:52:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.17.1/8.17.1) with ESMTPS id 325HqHc3089649 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 5 Mar 2023 19:52:20 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 325HqHc3089649 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 325HqHMe089648; Sun, 5 Mar 2023 19:52:17 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 5 Mar 2023 19:52:17 +0200 From: Konstantin Belousov To: David Chisnall Cc: Mateusz Guzik , freebsd-hackers Subject: Re: CFT: snmalloc as libc malloc Message-ID: References: <2f3dcda0-5135-290a-2dff-683b2e9fe271@FreeBSD.org> <2836EF0C-8B4B-441C-927B-3796457A3865@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Rspamd-Queue-Id: 4PV8S36xx1z3RDX X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Sun, Mar 05, 2023 at 05:27:16PM +0000, David Chisnall wrote: > Hi, > > Thanks for the comments. > > The FlagLock is used in two places: > > - Early init, before anything in libc can be guaranteed to exist. Before libc is initialized, there cannot be any threads except the current. Same for libthr. Since libthr uses internal allocator for things usually needed by mallocs, including internal mutexes and thread keys, the only duty of third-party malloc is to properly implement on-fork callbacks to ensure liveness after forks. From nobody Mon Mar 6 02:19:56 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVMjj1cS5z3wGZg for ; Mon, 6 Mar 2023 02:20:09 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVMjj0sNsz3CRt for ; Mon, 6 Mar 2023 02:20:09 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678069209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=VUWgXMNeOlowYkZBqfe2seHIBkNOLyB38Z1cHuWwuos=; b=DgcZd0dYnHcgeIIP7CYDbjSJjdeOJSCGqVHN0I76Qa5qoYjWAb08YXHmaZB3Zmurnl3e5P VtWjWR2e4jR8/ZktH/S1NQkxQcee/2vsr7ewhzCxaMJhLWmGvtwn9ut6FNwcTNhdo0gjET yeI5nLN5BDYuFZd/LeiWIlMjw4YSlAgkALYfI45W+bTMjbVtTVM1oRqkm33Ot1SCkNaG5h RkCpn0NTydfn0ZX/B1/EfQqe3G7l0S/InaNMxEaytlTKhL+x1QEZprs2AFcS86IjdhUBbX 5N6XUl/88iljO5xlVVy7cac7mHshFth20O/kjj/19xEgdXrzDpYBJY5409tw4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678069209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=VUWgXMNeOlowYkZBqfe2seHIBkNOLyB38Z1cHuWwuos=; b=hKVURnrJWk9WEVSr61NR6ivj793qUD7YEBQqWDAlFknIDk6wXp1GAW4RJsjKtTHsZu157D BB7/0AKL9ChYjNMkXc083dOOamp0QZsxzyNU25T2T53DWz3vdzUgaC0k+RVsMOvSB0ugHb aOf79ZJwgYVjkVafIC/mg6dbQw+zyrUs4NlXWMv7wLJmZRoj1fTQQ29e1tuyD6uo0jTAsP +GGmNHHnV9KtSeSi4aosEjFCQAa4MjZ4kjAQEFjH37NxB8WTY1PthTJa5sGsQc7FVvBjoM qkvjprEVYDE+94mrho3o4sTudYBcd3+zgRvovjWHyYdXHn7x6P01YlioLafwYw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678069209; a=rsa-sha256; cv=none; b=TDNexg4IpE3R4wE+Qbbk+OcE4vD+19FWLR2w9kz3Ouy5Hw5wt/QyimDXVqPxERajP1AMsk bKCPuDiVaQJUEedEHesnATrqnCcYrTFRwJfLMla3kq4Npq/xN//2nnb2vUFXyzejEz62JM +MTMrl1zc+AM/5PrY8LnTeq22N0U79Jhn+n1aRmETFz0NoIJzQAiPCvDnP0jQO1nUcpHuo ptfLmfhigMs2PJ6d0PF2EN9FyPe0hI3X8y8fu7V2o4HCbjS2PAQEBebgbyPSLkS5KIIhxS YH5AKVWnKpzZITQFQDo7KIFo6w8P4vUKPv+mCwg2LnRg0oc0nmLNPp6CX+9qOA== Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PVMjh6ynyztc8 for ; Mon, 6 Mar 2023 02:20:08 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qv1-f52.google.com with SMTP id y3so5726520qvn.4 for ; Sun, 05 Mar 2023 18:20:08 -0800 (PST) X-Gm-Message-State: AO0yUKXj3anArPsAepmeIg99ftsuPV381nr4Rxct5y6c2K41gmHnB40+ hrdcR3MhKsOJFMXuGNA8hp5dyOd14DaChsmzTKc= X-Google-Smtp-Source: AK7set/eLyN/X1iNYeR/gYIfrd9JqXYV9dsXMkfwtaWyD7nRYbOsoQHfsW06wjyEBC45bLe8Dt8p7osrtDOttHK2YtE= X-Received: by 2002:a05:6214:1854:b0:56e:8c9a:2610 with SMTP id d20-20020a056214185400b0056e8c9a2610mr2522038qvy.3.1678069208431; Sun, 05 Mar 2023 18:20:08 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Kyle Evans Date: Sun, 5 Mar 2023 20:19:56 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: acpi_cmbat with charge-limited battery To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-ThisMailContainsUnwantedMimeParts: N Hello! I've dealt with this mainly over the weekend, but my solution was to just disable acpi_cmbat entirely, which is maybe not the best solution but I can't tell if this should be considered a firmware bug or if it's something we could find a way to workaround in the kernel. Basically, I've set the firmware on my frame.work laptop to limit the battery charge to 80%. When it hits 80% while plugged in, things get a little funky- I assume it's because the firmware's trying to carefully maintain the limit, but I end up getting (at least) one acpi notification per second, alternating between BST_CHANGE/BIX_CHANGE, which in turn drives up CPU usage as we tap it out to devd and upowerd picks it up. upowerd ends up pegging a core consistently. Should we be rate-limiting these devd notifications? Is this even reasonable behavior for the firmware? I'm not really sure how other OS behave here, but I haven't really seen any complaints from other framework'ers. Thanks, Kyle Evans From nobody Mon Mar 6 03:33:14 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVPLF33s3z3wLGP for ; Mon, 6 Mar 2023 03:33:25 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVPLF051dz3Hnx for ; Mon, 6 Mar 2023 03:33:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52f.google.com with SMTP id ec29so2162109edb.6 for ; Sun, 05 Mar 2023 19:33:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; t=1678073602; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QKexbWDlgxqh+cCNCz5NpZeAg04tSie+/SFQ6YU4Ufk=; b=o1BGMaXUvBp3FgLooBLzF6AQdHSqFGlXt775d0Q0D/w2riiwDOYCaOfdYYLKfHDsmE IAiAHXhMFnRjIUAifq9AHWFz5eCru4hzpJAQaNE7JaSOJ/zdvBmkyzviPI2u8gEOwpEw Ig3TjrldDrp43mlnfb1D436ikIijGZL/d0KcVGNy5JbHvqCfRDgK2ePT9ZeRlQkpCPsS RWOXGiZiCSA8X8szlhI9wQmRi8wNDQk40J2djmLBLDBsQkSqs+ONrhkMZ9xPT0a+cMWo /Aki+7oWZXm2dWrYxSN4ftahOkgfIqFPSKBvXOq7jnZNjozO+nmB8zc3rch3T0yzF5BJ jxEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678073602; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QKexbWDlgxqh+cCNCz5NpZeAg04tSie+/SFQ6YU4Ufk=; b=vZhW5mDcPupNBXTZorT9eK3uPiFrhDcxXHzj9dVtZN7xip0dQKgqtwZUp8QNcX9rsS s+3JG7TWqT8mwxeR5uqwA6XPspUhglwYbc7kjrEbsJCYvsuL8J3bJCDNhQVu6WyZ+7G6 OUiUcKeEr/sX2YG+lqrlnWiXoZZ8w43OY1dbeTOWoCU5737YcgbFywiAM/Q2Pw5Svglw jyY6uJ4bO2ZlJ1IqaSGrvEg0aIOze9VzfwDD79G75hzJ3JZZ3U/uOhqTGGcOYQW5p1rd geBvqxD2D/htZYdOsAnDHG9v/GsyUe3GEKEV3S9lSp1t2JfKMJEPqicXsXqS9qNMJWBm NhZQ== X-Gm-Message-State: AO0yUKXGbo9zhn0SG+PVTP/+1T3Buj8QOCuWbF9S0Kd+h1WFtMgvi7r6 g5X/JhS3xoQjqI/55ZlkD284894qOwsBwMKQABkIu3G5u+8UI1Bm X-Google-Smtp-Source: AK7set9za/Pz/irS1pUbNYxj8hmx8hroicEIKV4MO1+37nRNdI312NWfmNzHMTiQRNhA6xL6Eq+aWKr5z6MyH8/Swkk= X-Received: by 2002:a50:f619:0:b0:4be:f5a0:a80a with SMTP id c25-20020a50f619000000b004bef5a0a80amr5022955edn.0.1678073601604; Sun, 05 Mar 2023 19:33:21 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sun, 5 Mar 2023 20:33:14 -0700 Message-ID: Subject: Re: acpi_cmbat with charge-limited battery To: Kyle Evans Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000ee54ad05f632f4fe" X-Rspamd-Queue-Id: 4PVPLF051dz3Hnx X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000ee54ad05f632f4fe Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Mar 5, 2023 at 7:20=E2=80=AFPM Kyle Evans wrot= e: > Hello! > > I've dealt with this mainly over the weekend, but my solution was to > just disable acpi_cmbat entirely, which is maybe not the best solution > but I can't tell if this should be considered a firmware bug or if > it's something we could find a way to workaround in the kernel. > > Basically, I've set the firmware on my frame.work laptop to limit the > battery charge to 80%. When it hits 80% while plugged in, things get a > little funky- I assume it's because the firmware's trying to carefully > maintain the limit, but I end up getting (at least) one acpi > notification per second, alternating between BST_CHANGE/BIX_CHANGE, > which in turn drives up CPU usage as we tap it out to devd and upowerd > picks it up. upowerd ends up pegging a core consistently. > > Should we be rate-limiting these devd notifications? Is this even > reasonable behavior for the firmware? I'm not really sure how other OS > behave here, but I haven't really seen any complaints from other > framework'ers. > Seems like this is crappy firmware behavior and we should rate limit in the driver... It's not useful information to be sharing once a second... Warner --000000000000ee54ad05f632f4fe Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sun, Mar 5, 2023 at 7:20=E2=80=AFP= M Kyle Evans <kevans@freebsd.org> wrote:
He= llo!

I've dealt with this mainly over the weekend, but my solution was to just disable acpi_cmbat entirely, which is maybe not the best solution
but I can't tell if this should be considered a firmware bug or if
it's something we could find a way to workaround in the kernel.

Basically, I've set the firmware on my frame.work laptop to limit the
battery charge to 80%. When it hits 80% while plugged in, things get a
little funky- I assume it's because the firmware's trying to carefu= lly
maintain the limit, but I end up getting (at least) one acpi
notification per second, alternating between BST_CHANGE/BIX_CHANGE,
which in turn drives up CPU usage as we tap it out to devd and upowerd
picks it up. upowerd ends up pegging a core consistently.

Should we be rate-limiting these devd notifications? Is this even
reasonable behavior for the firmware? I'm not really sure how other OS<= br> behave here, but I haven't really seen any complaints from other
framework'ers.

Seems like this is c= rappy firmware behavior and we should rate
limit in the driver...= It's not useful information to be sharing once
a second...

Warner
--000000000000ee54ad05f632f4fe-- From nobody Mon Mar 6 03:58:49 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVPvp72r4z3wMPC for ; Mon, 6 Mar 2023 03:59:02 +0000 (UTC) (envelope-from manav1811kumar@gmail.com) Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVPvp1M7fz3L1j; Mon, 6 Mar 2023 03:59:02 +0000 (UTC) (envelope-from manav1811kumar@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x62e.google.com with SMTP id a2so8867942plm.4; Sun, 05 Mar 2023 19:59:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678075140; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=RvWoDtWX5zylj3IuyQ1K22nO+WYSD6M7Iknqv43DtpU=; b=XJ12U6VqkBv9XMxGMIdRb0J1E/Yk47W6AI+MSZtzTPcfFlPSaUrkZYibqHZboi6p9q PMKXjPnxiBySxS5wHTQqE9nYe1K9VrDqJOg7px7YKycAQAcp0qXqhsX+GMQyphf8lnLU t3h4oEi3rHmBrw8wjwNOhyGYlCW58Vq5oaHTe6mnj0WAPPh3WJcPmHnrYnL+H+TNScXl Oa0y1+B/B2q5VZCECwU7XnNsHZAPEo5tNTEKum1ji5T5TypZ2ImErHTBVADYl/iRVaWB sPldXmRQWdZz+MPQ+cffGtcJGCbghXe2PMTl1s4y2Borq1uCUsg3hF/kP1TmLgk49pg/ +TJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678075140; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RvWoDtWX5zylj3IuyQ1K22nO+WYSD6M7Iknqv43DtpU=; b=TBWG+xnz2Br0r8jlYMAPmI87ihABMdSo6Bknc0oNK1kLSYE/bA9tx8Og7puMpOOf5i iJF7RpR+bueK7CnLjJ+9jBdL3YGFsKGCiUmjl2JvdK/rDocAMoGD2iVVpFXsMyOVFBuE vyu5zKRNRnWvz1jp6ZHruL7lDRKsJj3TETEI5O3sgGrBjAxDFqpKMJOEpCTQOz43o/jy fd/P64fHqTXo7va5IIcj5rHaB/aYI9We0QhMONtDx1h81ixmm9wEOrJnxBsP3f3q+yfD 6ujoxInn7FzC8N/6zF6EEkWFOCW120/xN8yhe+xGaUlyv3sSOj4Va3QgYIhvoOY6aecz m/TQ== X-Gm-Message-State: AO0yUKW5E3Kv/bgKQllz6wGUcbX2IWMbUjx686IyblRLt+irB2p37Q5v 6UP3pQKcZizSOlUIZiidB357jo7HBgi2xCnzIcNCZ07v X-Google-Smtp-Source: AK7set/yA/KpaiLO0oemw7GyO1+f7R6/TgqLnZzWV0EQFowNtDpQjgGkaKM3Yr8l9ekQehOZG0trNXy11vINkkyQfJE= X-Received: by 2002:a17:90b:1917:b0:23a:3636:f0af with SMTP id mp23-20020a17090b191700b0023a3636f0afmr3457979pjb.9.1678075140460; Sun, 05 Mar 2023 19:59:00 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: MANAV KUMAR Date: Mon, 6 Mar 2023 09:28:49 +0530 Message-ID: Subject: Re: acpi_cmbat with charge-limited battery To: Warner Losh Cc: Kyle Evans , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000a754c405f63350f8" X-Rspamd-Queue-Id: 4PVPvp1M7fz3L1j X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000a754c405f63350f8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Pls Unsubscribe me from this emailing list. Thanks On Mon, 6 Mar 2023 at 09:03, Warner Losh wrote: > > > On Sun, Mar 5, 2023 at 7:20=E2=80=AFPM Kyle Evans wr= ote: > >> Hello! >> >> I've dealt with this mainly over the weekend, but my solution was to >> just disable acpi_cmbat entirely, which is maybe not the best solution >> but I can't tell if this should be considered a firmware bug or if >> it's something we could find a way to workaround in the kernel. >> >> Basically, I've set the firmware on my frame.work laptop to limit the >> battery charge to 80%. When it hits 80% while plugged in, things get a >> little funky- I assume it's because the firmware's trying to carefully >> maintain the limit, but I end up getting (at least) one acpi >> notification per second, alternating between BST_CHANGE/BIX_CHANGE, >> which in turn drives up CPU usage as we tap it out to devd and upowerd >> picks it up. upowerd ends up pegging a core consistently. >> >> Should we be rate-limiting these devd notifications? Is this even >> reasonable behavior for the firmware? I'm not really sure how other OS >> behave here, but I haven't really seen any complaints from other >> framework'ers. >> > > Seems like this is crappy firmware behavior and we should rate > limit in the driver... It's not useful information to be sharing once > a second... > > Warner > --000000000000a754c405f63350f8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Pls Unsubscribe me from this emailing list.
Thanks

On Mon, 6 Mar 2023 at 09:03, Warner Losh <imp@bsdimp.com> wrote:


On Sun, Mar 5, 20= 23 at 7:20=E2=80=AFPM Kyle Evans <kevans@freebsd.org> wrote:
Hello!

I've dealt with this mainly over the weekend, but my solution was to just disable acpi_cmbat entirely, which is maybe not the best solution
but I can't tell if this should be considered a firmware bug or if
it's something we could find a way to workaround in the kernel.

Basically, I've set the firmware on my frame.work laptop to limit the
battery charge to 80%. When it hits 80% while plugged in, things get a
little funky- I assume it's because the firmware's trying to carefu= lly
maintain the limit, but I end up getting (at least) one acpi
notification per second, alternating between BST_CHANGE/BIX_CHANGE,
which in turn drives up CPU usage as we tap it out to devd and upowerd
picks it up. upowerd ends up pegging a core consistently.

Should we be rate-limiting these devd notifications? Is this even
reasonable behavior for the firmware? I'm not really sure how other OS<= br> behave here, but I haven't really seen any complaints from other
framework'ers.

Seems like this is c= rappy firmware behavior and we should rate
limit in the driver...= It's not useful information to be sharing once
a second...

Warner
--000000000000a754c405f63350f8-- From nobody Mon Mar 6 06:39:30 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVTTB06FBz3wVqJ for ; Mon, 6 Mar 2023 06:39:42 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVTT93wzBz3nWd for ; Mon, 6 Mar 2023 06:39:41 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678084781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3ZQqFsT9Yb7l0YmbhpCiKZ4lDDIF0Ydg2ALoalDV3VQ=; b=P1RZk5K3PIPAGUiVN92Tqgo30B4STyuoK1/Oaivl3s4xeZ4N2C9jp8f6wUq9G4mSVhynY/ q82dNzMUFkVBmw49z4txF2dmOl8T9++FU0AxbO1pzMmkj6jUVPwnyrXif2BmtK0b45K4iT uBN9F16XgHQMr8Z8QEci/FbuXKQAwSciA+XeqxMKWHJ1tR4gcv+dMu1+NtARr2AHz81yd0 VOOcmnywXuAKzXE8D5Pb/e+1Y3CBWIqu7laZi7zGpiGVJGAtw/xHbrqTPAh0FkSrBA+F8E 7Z1pHlM7Z1jjJuxIakRAukpdQssxP+hPY8OOJ3zps6luHH191P5bhSl/GLmOpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678084781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3ZQqFsT9Yb7l0YmbhpCiKZ4lDDIF0Ydg2ALoalDV3VQ=; b=G5QFTj6CG7zc4USwtcsfgfC3egcsnV9HXjmJjwuPUguu6Jd4Dgkhxs/JeFBzs58b1g3AqN 9jgRX7MBCHWhNxIAquwOTWYJPP/tb2pw+wKNxyMYMBo1a6ri3RfpzOhLl6xgznQkRgF+og QbchmkcIoIcKRMESDFx68YMalvyPwClQ74iwIsvWkEeQXCntizHCpm4X8MBpvlJnIhEM70 LPe94jZO9fvF8zg58IapmIhPspXUC1/y20Z8PirYOCBGhVSvgvdOZ5TXGg5k2kh0dZxIVP amMUnRmi32f8lbyXx5O580r8mpFsyoW/QMMh8iQIw70E8xOC/tneJ+779rQEcg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678084781; a=rsa-sha256; cv=none; b=DzPC89kSOD+2E0Qdyr4aBI+04XRQ45XEP7TezUOpRzPLz1SxSA/kTdNlRUzlKl8Z83zSAe LnTOQJ/xLxSRippl7ABe/Aq2hdSS5x5cUnBtqWvHj/jxNQ0ehpyHndg77VmXH2RHT36eUA 6B3mMW+jDCnbDmT/sznQXrknsJypiEMP0K8u82ou5N5MvjqS8lfJDdKpOjIpc7nKzC2H4M GxDNacRPJdILbKy/MD07MpcdPGePy91IjLcDS8Xz+sMlMGc3R1GZ0haUETOf+QtvydSMTI D0Mx26lxJX08xojIsJEOyNFhcXCvJAPqYQcOFTjxtaXHbq/GDz15eGF08Q5rEw== Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PVTT92q8mzxpX for ; Mon, 6 Mar 2023 06:39:41 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qt1-f169.google.com with SMTP id r16so7882571qtx.9 for ; Sun, 05 Mar 2023 22:39:41 -0800 (PST) X-Gm-Message-State: AO0yUKXGqRQLjhw7DlHJ1hZLwcf6PTZ5G3qq6jnlCtBN/mZCI+Ttcff9 Zu2Nry6y4iuH4eoJNZhbJbNGf/nXPgcwl1YdafY= X-Google-Smtp-Source: AK7set8onMkh4drf5CvIzH59qs53W1jdV1He3aUR4N04o9G/vgk/kJ2zvIaU83VtIIAJA5J5sA2s95nRaKRJC4iBK+g= X-Received: by 2002:ac8:40cb:0:b0:3bf:c33e:93a9 with SMTP id f11-20020ac840cb000000b003bfc33e93a9mr2772959qtm.1.1678084780915; Sun, 05 Mar 2023 22:39:40 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Kyle Evans Date: Mon, 6 Mar 2023 00:39:30 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: acpi_cmbat with charge-limited battery To: Warner Losh Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-ThisMailContainsUnwantedMimeParts: N On Sun, Mar 5, 2023 at 9:33=E2=80=AFPM Warner Losh wrote: > > > > On Sun, Mar 5, 2023 at 7:20=E2=80=AFPM Kyle Evans wr= ote: >> >> Hello! >> >> I've dealt with this mainly over the weekend, but my solution was to >> just disable acpi_cmbat entirely, which is maybe not the best solution >> but I can't tell if this should be considered a firmware bug or if >> it's something we could find a way to workaround in the kernel. >> >> Basically, I've set the firmware on my frame.work laptop to limit the >> battery charge to 80%. When it hits 80% while plugged in, things get a >> little funky- I assume it's because the firmware's trying to carefully >> maintain the limit, but I end up getting (at least) one acpi >> notification per second, alternating between BST_CHANGE/BIX_CHANGE, >> which in turn drives up CPU usage as we tap it out to devd and upowerd >> picks it up. upowerd ends up pegging a core consistently. >> >> Should we be rate-limiting these devd notifications? Is this even >> reasonable behavior for the firmware? I'm not really sure how other OS >> behave here, but I haven't really seen any complaints from other >> framework'ers. > > > Seems like this is crappy firmware behavior and we should rate > limit in the driver... It's not useful information to be sharing once > a second... > The more I think about it (and with your generally confirmatory response), the more I think approaching both problems independently is probably good. I added a quick sysctl to acpi_cmbat to allow an interval for folks with broken firmware like myself; I'll probably throw that up for review tomorrow-ish. I'll also reach out to frame.work folks and see if they can improve this in some way, but I suspect any action there will take a while. Thanks, Kyle Evans From nobody Mon Mar 6 06:51:42 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVTlH68hKz3wWQ8 for ; Mon, 6 Mar 2023 06:51:55 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVTlH2Gn3z3qF7 for ; Mon, 6 Mar 2023 06:51:55 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x835.google.com with SMTP id r5so9542623qtp.4 for ; Sun, 05 Mar 2023 22:51:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1678085514; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8GXc/rV6Gr6APNeMVwe2/jGL8qN1dqyw7BWEWugvIv8=; b=cUe1WtxIvJvLJrYRxqJ9ef0Ap3LpV1dHMNoCBy9fRClqfKxOlaG3tt5vNtZIHnxUTk VdAQXyxJS+UYTXIRWqeycZbfKVjYMO7TnQtwEeTj/SnGDGgtxLiGexI0Tx0rOsHccHK5 hU8lxQ3QKpBwkwaX+r6Dy5m6XdH8R7PUnhe6Qa2s2ZwU/vg4DCfSDBfPpIeZ5OBW6K+0 SIFtGmMF0SgvQa2QlRSB2U6/BnBITXMxrl7eYL+ML/JX/rLksRErzzpmLusFg7hMykSx uZUKyi0vwzkvsjbuyK9f3wugB/h73euU6WSBzimgCHyFbfk2K0O/+NNa9py5tD+TOHrH 2XEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678085514; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8GXc/rV6Gr6APNeMVwe2/jGL8qN1dqyw7BWEWugvIv8=; b=BAP1TZAOAp3ztKOhQPPQGofkslkPGAMsxK5BbI4Im4zJfRMBqEEUfSgbBMdOQQNAKa fqF4o6YJiBRigMH/FmULOypvBI58851HGrOhPtJv55pqUFsv98HKsbA+RScLIjWalb2B Qz72PPW3TwKjF3K/ii7MKVRXbFyfH1JvtuDvHgqnqpzBWXP8JXr8goqVkha8Jy1j9YGj 8YKrtmGVvR8GJVe87r2MxHFbMTK2vTAjVRO6motoy2CDen0E9adR4zw6gfX6k/D+46I9 2TNWXmI6dr9coC52UGO4LYTGgZddD0rvX2U0ngv2FQ9bs6fCCIREWVsyJG8k/vzxpsSz WZRQ== X-Gm-Message-State: AO0yUKWprxlpjpAdPNxWhsxtyhfVqabVTaKRUnVgBdoKSnXxtqrAJ1K0 NTCIHKMt7draNP2K+K5LAI1M/w== X-Google-Smtp-Source: AK7set9X4gqrdjHG5Y9bRBlmYhp9U1ozberYmTzEeJ37iUW8Oq3bEvmApADl3A52OrYkIan/pFel5A== X-Received: by 2002:a05:622a:130e:b0:3b9:b6b7:90c5 with SMTP id v14-20020a05622a130e00b003b9b6b790c5mr15695584qtk.49.1678085514379; Sun, 05 Mar 2023 22:51:54 -0800 (PST) Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com. [209.85.219.176]) by smtp.gmail.com with ESMTPSA id m18-20020ac866d2000000b003bfbfd9a4aesm6994473qtp.56.2023.03.05.22.51.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 05 Mar 2023 22:51:54 -0800 (PST) Received: by mail-yb1-f176.google.com with SMTP id t39so7093880ybi.3; Sun, 05 Mar 2023 22:51:53 -0800 (PST) X-Received: by 2002:a5b:bc6:0:b0:a0d:8150:be04 with SMTP id c6-20020a5b0bc6000000b00a0d8150be04mr4545792ybr.13.1678085513610; Sun, 05 Mar 2023 22:51:53 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Tomek CEDRO Date: Mon, 6 Mar 2023 07:51:42 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: acpi_cmbat with charge-limited battery To: Kyle Evans Cc: Warner Losh , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4PVTlH2Gn3z3qF7 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Mon, Mar 6, 2023 at 7:39=E2=80=AFAM Kyle Evans wrot= e: > On Sun, Mar 5, 2023 at 9:33=E2=80=AFPM Warner Losh wrote= : > > On Sun, Mar 5, 2023 at 7:20=E2=80=AFPM Kyle Evans = wrote: > >> > >> Hello! > >> > >> I've dealt with this mainly over the weekend, but my solution was to > >> just disable acpi_cmbat entirely, which is maybe not the best solution > >> but I can't tell if this should be considered a firmware bug or if > >> it's something we could find a way to workaround in the kernel. > >> > >> Basically, I've set the firmware on my frame.work laptop to limit the > >> battery charge to 80%. When it hits 80% while plugged in, things get a > >> little funky- I assume it's because the firmware's trying to carefully > >> maintain the limit, but I end up getting (at least) one acpi > >> notification per second, alternating between BST_CHANGE/BIX_CHANGE, > >> which in turn drives up CPU usage as we tap it out to devd and upowerd > >> picks it up. upowerd ends up pegging a core consistently. > >> > >> Should we be rate-limiting these devd notifications? Is this even > >> reasonable behavior for the firmware? I'm not really sure how other OS > >> behave here, but I haven't really seen any complaints from other > >> framework'ers. > > > > Seems like this is crappy firmware behavior and we should rate > > limit in the driver... It's not useful information to be sharing once > > a second... > > The more I think about it (and with your generally confirmatory > response), the more I think approaching both problems independently is > probably good. I added a quick sysctl to acpi_cmbat to allow an > interval for folks with broken firmware like myself; I'll probably > throw that up for review tomorrow-ish. I'll also reach out to > frame.work folks and see if they can improve this in some way, but I > suspect any action there will take a while. Maybe some sort of PID function would help separate data and control? :-) https://en.wikipedia.org/wiki/PID_controller --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Mon Mar 6 10:13:24 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVZCs1bX2z3wjPP for ; Mon, 6 Mar 2023 10:13:29 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVZCq6THYz4G5H for ; Mon, 6 Mar 2023 10:13:27 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm1 header.b=fguEgi17; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=oj2Ue0er; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.29 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id C0FE15C0254 for ; Mon, 6 Mar 2023 05:13:26 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 06 Mar 2023 05:13:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm1; t=1678097606; x=1678184006; bh=H6d8L0eGm+e9lddW3wLAZPFro CqkedufsWaG+KWv9GE=; b=fguEgi17gPszOxGBDXhOLaF+MYKBUQ4n5OtojQT4x mff78BBu1y5juwFsmytpJz0E8OcS5IXGd+IRXclM+Lo4C6d46a5ej76mdSq7NWAX zHq0lMxtwe/KqtYH/TgaLEf9kG4NPnNHGym/InCvJ3YTcdpp7fTaSgeINyNvALSR pANLfKZWiKmQohkUf4HGV2adhorFd3BSLaL2FFIyTwJOIBI4vgUKvrGEtSMsEzMj S6QdbKLEmzlgkI27YMANXFfRIEJWfblvf7Sssd6f9A2ZG4VzSzOS3E7Wo4hJhQSN hFdDGk4QCWaEtmzsqoLmVX5SdQ+h9geNhEF6ELMEBhA7w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1678097606; x=1678184006; bh=H6d8L0eGm+e9lddW3wLAZPFroCqkedufsWa G+KWv9GE=; b=oj2Ue0erZejF2i1Wbj+o1cCgSMyKy7ETjz2+HaWgcQkNieXT7DQ cz+1vjfin5VnWDqb+Yn7UWe4BygyhmnTwtQ7l19ztntsbmtw75CMl5PidXWCK3hx EwK7JkBQTf6ftxS59sP7/WgMbPMEejPaNAZP9uP3Hz3eqykattbM3PAeyCXlyu+L WtTVtT5DMvX+gdypZdVSS4Te+3+w7W8kALP6treQ/u4LR5x/lmtZBt5YTlHUgy1A TW4ICuaej49PQ+WhXgOSfrKcIMsj+1oUbIZteP+iGSzipoG517LxQU0ZxGVwXtpV PXNO6vsQ+2V4pLDyWhQdiHG5d8SnMKgZJGQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvddtiedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtugesthdtredttd dtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgrthht vghrnhepveduffeivdfffffghfegfeejfefftdeiteehteekfefhvdefgfettdeuheegff eunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepvhho ihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 6 Mar 2023 05:13:26 -0500 (EST) Date: Mon, 6 Mar 2023 10:13:24 +0000 From: void To: freebsd-hackers@freebsd.org Subject: kern.elf64.allow_wx=0 and bhyve host and guest Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline X-Spamd-Result: default: False [-4.58 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.981]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm1,messagingengine.com:s=fm1]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4PVZCq6THYz4G5H X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N Hello hackers@ If kern.elf64.allow_wx=0 on a bhyve host, will its effects be apparent in a bhyve (freebsd) guest? tia, -- From nobody Mon Mar 6 14:49:30 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVhLb1HZQz3wytM for ; Mon, 6 Mar 2023 14:49:43 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVhLZ4HMfz3l4Z; Mon, 6 Mar 2023 14:49:42 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Authentication-Results: mx1.freebsd.org; none Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 326EnUjx078467; Mon, 6 Mar 2023 06:49:30 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 326EnUrV078466; Mon, 6 Mar 2023 06:49:30 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202303061449.326EnUrV078466@gndrsh.dnsmgr.net> Subject: Re: acpi_cmbat with charge-limited battery In-Reply-To: To: Kyle Evans Date: Mon, 6 Mar 2023 06:49:30 -0800 (PST) CC: Warner Losh , FreeBSD Hackers X-Mailer: ELM [version 2.4ME+ PL121h (25)] List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4PVhLZ4HMfz3l4Z X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > On Sun, Mar 5, 2023 at 9:33?PM Warner Losh wrote: > > > > > > > > On Sun, Mar 5, 2023 at 7:20?PM Kyle Evans wrote: > >> > >> Hello! > >> > >> I've dealt with this mainly over the weekend, but my solution was to > >> just disable acpi_cmbat entirely, which is maybe not the best solution > >> but I can't tell if this should be considered a firmware bug or if > >> it's something we could find a way to workaround in the kernel. > >> > >> Basically, I've set the firmware on my frame.work laptop to limit the > >> battery charge to 80%. When it hits 80% while plugged in, things get a > >> little funky- I assume it's because the firmware's trying to carefully > >> maintain the limit, but I end up getting (at least) one acpi > >> notification per second, alternating between BST_CHANGE/BIX_CHANGE, > >> which in turn drives up CPU usage as we tap it out to devd and upowerd > >> picks it up. upowerd ends up pegging a core consistently. > >> > >> Should we be rate-limiting these devd notifications? Is this even > >> reasonable behavior for the firmware? I'm not really sure how other OS > >> behave here, but I haven't really seen any complaints from other > >> framework'ers. > > > > > > Seems like this is crappy firmware behavior and we should rate > > limit in the driver... It's not useful information to be sharing once > > a second... > > I agree with that assesment of the firmware, but it brings up a question of is the firmware actually doing just what it is told? I see above an upper limit on the charge has been set at 80%, but I do not see any mention of a "resume charging" after the battery drops to XX%. I assume what is happening is the charge stops at 80%, we drop some load on the battery and it quickly drops to 79%, and the firmware decides it must start to charge again. Kyle, does the bios have settings for a resume charging? Also might be intereting to watch the battery level in a tight loop to see if you can catch the drop. > > The more I think about it (and with your generally confirmatory > response), the more I think approaching both problems independently is > probably good. I added a quick sysctl to acpi_cmbat to allow an > interval for folks with broken firmware like myself; I'll probably > throw that up for review tomorrow-ish. I'll also reach out to > frame.work folks and see if they can improve this in some way, but I > suspect any action there will take a while. > > Thanks, > > Kyle Evans > > > -- Rod Grimes rgrimes@freebsd.org From nobody Tue Mar 7 03:57:04 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PW1qJ0Fl3z3wqBF for ; Tue, 7 Mar 2023 03:57:16 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PW1qH6x3Cz43cL; Tue, 7 Mar 2023 03:57:15 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678161436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qDaeAOeNR//gjvTPyOLmDcWJsXuRax7uM9P4m/vugTs=; b=ynObWNmQuxm/0LFchaon1+c6leohMVxMqy774FB9uOLJe7ndX3pk9qRa5Z2lP3LwVFHB/n ZO0fMRccs2jWS06LqUO+p/GR2apghfa4HKtFOWPCo5tfAiGXGwBytf4cRwIz+vRin8xD9e FIZR0r/306j7JCjVHaXcb3foUNfxyPoSLjuMXVCEAXzNIEiW4u1k/JHBmxaVWktoKTr1Ai vJXkLoYrprebQzxz/1VMsr/Bo7Kk3Ljn+wNNq/vkKyFuo2o/SrDxwOoqnzvYN5Y2jgaat1 KbhhS26fdapvtWL7N6byF4ce1kiIvwS7cF7vY/8tKH8gmo6UBSigpfQcWIavkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678161436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qDaeAOeNR//gjvTPyOLmDcWJsXuRax7uM9P4m/vugTs=; b=v7pb8MlqubQ8PRwj3NiHm19iyPHSDytcyN7/fwWbY//v8UfRL+XiYBo6bS60Sq1KNai39r duOZV39QBz7NFZtw56I3JVwq74DNP+yxSH5sQqcp/E3QStylOtScvXlLjutTus5Md32f/N 6Sg1a0r88l+GzdU1fSEPcnXT58zsiLgpz8YuaH14k8d1piwC6tWGqcmdXWvcdVS2MM4H0Q 9l41QKFKFP15hdnjH/3I9gWimZfNak1Zb/MMbmOAo7Zi6k0vWScOFqSXNiKhVtMCnEmV2e lz6zgMpL/PEQiNZaqrJX7f4180Pi2XqnHtTv9WsMe5In20Pl0O4o1cv/GKzLPw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678161436; a=rsa-sha256; cv=none; b=gGgXJ114OvB6n4OvpVHj56MA1bx/GaZt79IBZ/hJvDy3RRSlzuf+Ay6VBNASEj9RnKH3Ud 7GcUJANIz8UbND7sLva4ZokByWU0dlRXiNm2BtvBA4SYnBfroeNXPcgECpe9c2vXb4SRcM tlouJ989sRJHEZyz3PRt7ULF4Jakk2im2LBLsVM2uQ2G95PHCmz+l38pq5A0zNrMPBog75 vToN/tYRlpaPzXjSYFEJfQ6y0Om3RQW/NCV4WQteH3RgF/+iN2Y4mGfaXK6F2eFlzIOmxK n5TwxKgW7uWS1/ZDX6Ynyv29X/YFeUfPoGQ2cS6oYpyvI/mgwa8uMiP2/KuJIA== Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PW1qH5pnpzLg2; Tue, 7 Mar 2023 03:57:15 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qv1-f54.google.com with SMTP id g9so6475498qvt.8; Mon, 06 Mar 2023 19:57:15 -0800 (PST) X-Gm-Message-State: AO0yUKVFlEbISvWXUAun45dkxejYTXah2YgFi8fICIrFeC5saAIXgYJm aRTAqpXCVTV3uTQxpOIvuU94EoKTRdEYLU2Vw2g= X-Google-Smtp-Source: AK7set+HCcB2ERDANS6x5jFda0ZjqGNEfmRLTPU1mB6Nbu5lqg+NiZsBSBJQsPURLT9sAwHk2+ZJTg+sCs8hHNUT7kA= X-Received: by 2002:a05:6214:1752:b0:56c:1865:feb with SMTP id dc18-20020a056214175200b0056c18650febmr3867550qvb.3.1678161435455; Mon, 06 Mar 2023 19:57:15 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <202303061449.326EnUrV078466@gndrsh.dnsmgr.net> In-Reply-To: <202303061449.326EnUrV078466@gndrsh.dnsmgr.net> From: Kyle Evans Date: Mon, 6 Mar 2023 21:57:04 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: acpi_cmbat with charge-limited battery To: "Rodney W. Grimes" Cc: Kyle Evans , Warner Losh , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-ThisMailContainsUnwantedMimeParts: N On Mon, Mar 6, 2023 at 8:49=E2=80=AFAM Rodney W. Grimes wrote: > > > On Sun, Mar 5, 2023 at 9:33?PM Warner Losh wrote: > > > > > > > > > > > > On Sun, Mar 5, 2023 at 7:20?PM Kyle Evans wrote: > > >> > > >> Hello! > > >> > > >> I've dealt with this mainly over the weekend, but my solution was to > > >> just disable acpi_cmbat entirely, which is maybe not the best soluti= on > > >> but I can't tell if this should be considered a firmware bug or if > > >> it's something we could find a way to workaround in the kernel. > > >> > > >> Basically, I've set the firmware on my frame.work laptop to limit th= e > > >> battery charge to 80%. When it hits 80% while plugged in, things get= a > > >> little funky- I assume it's because the firmware's trying to careful= ly > > >> maintain the limit, but I end up getting (at least) one acpi > > >> notification per second, alternating between BST_CHANGE/BIX_CHANGE, > > >> which in turn drives up CPU usage as we tap it out to devd and upowe= rd > > >> picks it up. upowerd ends up pegging a core consistently. > > >> > > >> Should we be rate-limiting these devd notifications? Is this even > > >> reasonable behavior for the firmware? I'm not really sure how other = OS > > >> behave here, but I haven't really seen any complaints from other > > >> framework'ers. > > > > > > > > > Seems like this is crappy firmware behavior and we should rate > > > limit in the driver... It's not useful information to be sharing once > > > a second... > > > > > I agree with that assesment of the firmware, but it brings up > a question of is the firmware actually doing just what it is > told? I see above an upper limit on the charge has been > set at 80%, but I do not see any mention of a "resume charging" > after the battery drops to XX%. I assume what is happening is > the charge stops at 80%, we drop some load on the battery and > it quickly drops to 79%, and the firmware decides it must > start to charge again. > > Kyle, does the bios have settings for a resume charging? > Also might be intereting to watch the battery level in > a tight loop to see if you can catch the drop. > I haven't found any such setting, even after my last update to a BIOS from ~a couple months ago. I tried `acpiconf -i0` in a tight loop and captured the below transition. Interestingly, I wasn't able to capture anywhere where it was charging and reported as < 80%, but for all I know that's a rounding error and there's just not enough tolerance to catch the difference. Design capacity: 3572 mAh Last full capacity: 3503 mAh Technology: secondary (rechargeable) Design voltage: 15400 mV Capacity (warn): 350 mAh Capacity (low): 105 mAh Cycle Count: 5 Measurement Accuracy: 0 % Max Sampling Time: 0 ms Min Sampling Time: 0 ms Max Average Interval: 0 ms Min Average Interval: 0 ms Low/warn granularity: 264 mAh Warn/full granularity: 3780 mAh Model number: Framewo Serial number: 0260 Type: LION OEM info: NVT State: discharging Remaining capacity: 80% Remaining time: 706:15 Present rate: 4 mA (66 mW) Present voltage: 16594 mV Design capacity: 3572 mAh Last full capacity: 3503 mAh Technology: secondary (rechargeable) Design voltage: 15400 mV Capacity (warn): 350 mAh Capacity (low): 105 mAh Cycle Count: 5 Measurement Accuracy: 0 % Max Sampling Time: 0 ms Min Sampling Time: 0 ms Max Average Interval: 0 ms Min Average Interval: 0 ms Low/warn granularity: 264 mAh Warn/full granularity: 3780 mAh Model number: Framewo Serial number: 0260 Type: LION OEM info: NVT State: charging Remaining capacity: 80% Remaining time: unknown Present rate: 4 mA (66 mW) Present voltage: 16600 mV From nobody Tue Mar 7 14:10:21 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PWHQq2XTcz3wBkx for ; Tue, 7 Mar 2023 14:10:27 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PWHQp5vbXz4159; Tue, 7 Mar 2023 14:10:26 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Authentication-Results: mx1.freebsd.org; none Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 327EALfo082541; Tue, 7 Mar 2023 06:10:21 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 327EALkt082540; Tue, 7 Mar 2023 06:10:21 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202303071410.327EALkt082540@gndrsh.dnsmgr.net> Subject: Re: acpi_cmbat with charge-limited battery In-Reply-To: To: Kyle Evans Date: Tue, 7 Mar 2023 06:10:21 -0800 (PST) CC: "Rodney W. Grimes" , Warner Losh , FreeBSD Hackers X-Mailer: ELM [version 2.4ME+ PL121h (25)] List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4PWHQp5vbXz4159 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > On Mon, Mar 6, 2023 at 8:49?AM Rodney W. Grimes > wrote: > > > > > On Sun, Mar 5, 2023 at 9:33?PM Warner Losh wrote: > > > > > > > > > > > > > > > > On Sun, Mar 5, 2023 at 7:20?PM Kyle Evans wrote: > > > >> > > > >> Hello! > > > >> > > > >> I've dealt with this mainly over the weekend, but my solution was to > > > >> just disable acpi_cmbat entirely, which is maybe not the best solution > > > >> but I can't tell if this should be considered a firmware bug or if > > > >> it's something we could find a way to workaround in the kernel. > > > >> > > > >> Basically, I've set the firmware on my frame.work laptop to limit the > > > >> battery charge to 80%. When it hits 80% while plugged in, things get a > > > >> little funky- I assume it's because the firmware's trying to carefully > > > >> maintain the limit, but I end up getting (at least) one acpi > > > >> notification per second, alternating between BST_CHANGE/BIX_CHANGE, > > > >> which in turn drives up CPU usage as we tap it out to devd and upowerd > > > >> picks it up. upowerd ends up pegging a core consistently. > > > >> > > > >> Should we be rate-limiting these devd notifications? Is this even > > > >> reasonable behavior for the firmware? I'm not really sure how other OS > > > >> behave here, but I haven't really seen any complaints from other > > > >> framework'ers. > > > > > > > > > > > > Seems like this is crappy firmware behavior and we should rate > > > > limit in the driver... It's not useful information to be sharing once > > > > a second... > > > > > > > > I agree with that assesment of the firmware, but it brings up > > a question of is the firmware actually doing just what it is > > told? I see above an upper limit on the charge has been > > set at 80%, but I do not see any mention of a "resume charging" > > after the battery drops to XX%. I assume what is happening is > > the charge stops at 80%, we drop some load on the battery and > > it quickly drops to 79%, and the firmware decides it must > > start to charge again. > > > > Kyle, does the bios have settings for a resume charging? > > Also might be intereting to watch the battery level in > > a tight loop to see if you can catch the drop. > > > > I haven't found any such setting, even after my last update to a BIOS > from ~a couple months ago. I tried `acpiconf -i0` in a tight loop and > captured the below transition. Interestingly, I wasn't able to capture > anywhere where it was charging and reported as < 80%, but for all I > know that's a rounding error and there's just not enough tolerance to > catch the difference. The fact that your able to capture it as both charging and discharging at an 80% full capacity leads me to believe the firmware has no hysterisis around this battery full setting and is infact flipping the charger on and off every second in an attempt to keep the battery at that 80% level. A comment inline with the battery reports below. > > Design capacity: 3572 mAh > Last full capacity: 3503 mAh > Technology: secondary (rechargeable) > Design voltage: 15400 mV > Capacity (warn): 350 mAh > Capacity (low): 105 mAh > Cycle Count: 5 > Measurement Accuracy: 0 % > Max Sampling Time: 0 ms > Min Sampling Time: 0 ms > Max Average Interval: 0 ms > Min Average Interval: 0 ms > Low/warn granularity: 264 mAh Wow, that is a 7% step size. I am more use to seeing this as some double digit value near 1/100 of "Design Capacity". For this battery I would expect to be seeing 35mAh or so. > Warn/full granularity: 3780 mAh Wait a minute, how can this granularity be larger than the total battery size? I am use to seeing this value as the same as Low/warn granularity. > Model number: Framewo > Serial number: 0260 > Type: LION > OEM info: NVT > State: discharging > Remaining capacity: 80% > Remaining time: 706:15 > Present rate: 4 mA (66 mW) This number also seems bonkers, there is no way no laptop running FreeBSD is doing so on 66mW of power!!!! That wouldnt even power up an LED backlight of any reasonable brightness. Perhaps get a new set of "Discharging" values with the AC adapter un plugged and the battery discharging for at least a minute. > Present voltage: 16594 mV > > Design capacity: 3572 mAh > Last full capacity: 3503 mAh > Technology: secondary (rechargeable) > Design voltage: 15400 mV > Capacity (warn): 350 mAh > Capacity (low): 105 mAh > Cycle Count: 5 > Measurement Accuracy: 0 % > Max Sampling Time: 0 ms > Min Sampling Time: 0 ms > Max Average Interval: 0 ms > Min Average Interval: 0 ms > Low/warn granularity: 264 mAh > Warn/full granularity: 3780 mAh > Model number: Framewo > Serial number: 0260 > Type: LION > OEM info: NVT > State: charging > Remaining capacity: 80% > Remaining time: unknown > Present rate: 4 mA (66 mW) Again this nuber looks bonkers, charging at this power level makes no since at all. > Present voltage: 16600 mV > > -- Rod Grimes rgrimes@freebsd.org From nobody Wed Mar 8 09:19:59 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PWmxG5Sjjz3w8kk for ; Wed, 8 Mar 2023 09:20:02 +0000 (UTC) (envelope-from obiwac@gmail.com) Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PWmxF6BsNz3whX for ; Wed, 8 Mar 2023 09:20:01 +0000 (UTC) (envelope-from obiwac@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=PyxbRPcJ; spf=pass (mx1.freebsd.org: domain of obiwac@gmail.com designates 2a00:1450:4864:20::536 as permitted sender) smtp.mailfrom=obiwac@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-x536.google.com with SMTP id cw28so62959730edb.5 for ; Wed, 08 Mar 2023 01:20:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678267200; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=RlRGajhEgHO4zWlVngcYSzuslkjiwMm5t37wsoH1Lqc=; b=PyxbRPcJnwpvw3SF4YXIAkzfApa2p96u/b6MlMmhSQG4IpEZMQ2YtaOGR8lpWZ3y6q eoyap2UNi5sGcQFDGBSAFD4dKqapilIFk2Jc95vZ8wEBlovzoUKyDjehzCQh/4/Wn1En bPGRTKKT9BQEs9NhUTyEHQxg43iX50AIp4RU7yI9YqzjTOkZb2qMOWFNELQPBTYE0INj bPoJMCpnFxkliDsAZwezwAuWzslemn0eNLfCjNJoNnPxs5mJa1qT4rto0V2AuIc/5Cnb ypjZkDLk6kOzQ9YJbFNrL96j8fwYF6GtVCt0awTPXT2qNQMEBlG9rOr1T2C2IqxjwkNR IOHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678267200; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=RlRGajhEgHO4zWlVngcYSzuslkjiwMm5t37wsoH1Lqc=; b=buq5TNcAR67QkMyLF3XuneGk1U9RWqA2nuDJYxRGMomD6N5Wm3RLuLm1NTxyihul2A lQBL021pSnTQO70uFWzFB5jBF6H38jA6uAS5m900KCnEMUF0JHyJJvX3wkYwUeCjgH4b mv6/PWusfp74m8BRl7MfGsKe7GITwPHLJOfm3PLlf/gU8n2EO+W0zGvgwgGgGat7vzlZ Iqanxed+Yq4QzeYRwyKDz5N0mQkdp3vCdRa/X7HJi031rGqBqCHpCyFpag487j26n7Ko 2ucS38ZZKj0ZjZkLc29SvnP3ItmgBPP6XN1gj+9Y/SONww/HnO1egOdwaptYPapD+0+W ppQw== X-Gm-Message-State: AO0yUKUJ2gsWIkalZXGuqPGtZVZ3lBM5albxtnqh7wR5BS2QU9Hindz7 jPn7ViG7HDv2+8EsPMK567D0IApLGb4DSct1Ftr8eZcT X-Google-Smtp-Source: AK7set8ETVTqrseFi2t85gwGC0jKM7RIn8CHSYhJvlkIZtREPBrZMhIGcbE9a7I4Gd5fcu+l3Y7NqfmVaJzVjjyn3Kk= X-Received: by 2002:a50:d615:0:b0:4bb:c90e:1fa5 with SMTP id x21-20020a50d615000000b004bbc90e1fa5mr9456302edi.8.1678267199845; Wed, 08 Mar 2023 01:19:59 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a17:907:c011:b0:8e7:7343:cad1 with HTTP; Wed, 8 Mar 2023 01:19:59 -0800 (PST) From: obiwac Date: Wed, 8 Mar 2023 10:19:59 +0100 Message-ID: Subject: GSoC 2023 project ideas To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-3.17 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_SHORT(-0.17)[-0.175]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::536:from]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4PWmxF6BsNz3whX X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Hi! I'm a CS student from Belgium and I'm interested in participating in Google Summer of Code as a FreeBSD contributor. I've been using FreeBSD for a bit over two years and I've already upstreamed a few patches over that time, but I'd like to give back in a little more of a meaningful way :) More specifically, I was thinking of proposing work on the Linux KPI for compiling network drivers on FreeBSD once the applications open (see SummerOfCodeIdeas: https://wiki.freebsd.org/SummerOfCodeIdeas#Add_support_for_building_Linux-only_network_drivers). I see netlink was the subject of a previous SoC project by Sean Ng (and their mentor melifaro@), so it would seem at least some part of this has already been done. Would someone potentially be interested in mentoring such a project (perhaps mmokhi@ or melifaro@)? Is there maybe something you'd like to add on top of what's currently on the ideas page wrt to this? I'm posting this message is mostly to gauge overall interest in this idea in particular or if I'd be better off focusing my efforts on something else... Personally I'd be very interested in working on this if only for personal use, but maybe there's something I'd be equally interested in which is more useful to more people ;) Kind regards, Aymeric From nobody Wed Mar 8 11:39:17 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PWr293j8Rz3wHTt for ; Wed, 8 Mar 2023 11:39:29 +0000 (UTC) (envelope-from melifaro@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PWr293711z4Dh7; Wed, 8 Mar 2023 11:39:29 +0000 (UTC) (envelope-from melifaro@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678275569; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7KCOI1DoQiWaGpUw0GHfLnKQyrWRNlX+MDS1yKUjeW8=; b=pwgr+Ugv0fVlIvZylHnjlHKhlGcs+9LfqGRqCybsMrh6HPEdnZyhd1KMscoXSILNZI1YWb mJxAy7NjJ83JXs3rox8zGLIMNeI/5oq+u7vkl6W5dIJ970YExCQRWnEafG8uiOTsngqcul sGTMoE68S6MWeAUOlsV8QMAuIU5DZKGmET0w3qYbtfZbMTZHueehFAWBmOxalto2T6II1g 0rADT1wQvvJFiDYT6STrEWXiMOZPRbpDHL+G8Ekxh4l0Nag9T9ir67r3wVf73owcgZCp00 gIDNkUJx/AMvOTJ6uTAgEGiNmMb2p06sK4AAZckxHrINNYEZ0I21OK9GaRssJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678275569; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7KCOI1DoQiWaGpUw0GHfLnKQyrWRNlX+MDS1yKUjeW8=; b=V2cCoehCGXefDVNS6Ds5Jdh8LDm6eE01Zr9s1diHiEN21AqYbwA4iIZeLhS0cr9XqbyRHD hb8Bac3J3D8s4vMiBgDeC4C6y6sqJAdTjhiPRgekziNGGcrAw7yqGMSyqELS9owZwEjzF2 ADrDSurqy/OPgKbKmfHiqqc/r/KuodP2SwJI0g+/XaIxmYk+EQaaezmLEba0G0opy/Qy8h 7P9hUr33kjnXphYtfdQfDVZYwognX5FfD4xBNUYVBggyEnYYN51JaEurqx7UrJn+VNhqn5 rSqeUVSTyA8PpYhHhGoxbBPfvIO86jw3sOWO8/Ydl1n9kCqZqICJilNgkZJg9A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678275569; a=rsa-sha256; cv=none; b=d4D0ahcp5y4sUw88wcxJiPljfZXpmulPdOE4ha+NzS28t5Bzr0TpheNTw0JN6+Osm/Vgkv c2IpWk1Rmd8nl4C52zk4VcwpIA4/2ANc0OfLWRinA7zEfhtKTQD/KrVG/fEgw8aooELWkB z/Nm09GkoHlDY2iLSrdVLgG4umkfEykAVdbkxyJh0amvZEa6rk8AoqIFq2PGheDLb5v9yY 9Z3GD8sGz2qZLioBAzViNDxxNwlfjcRoVHqQhb0xhwQKS9DbyzwYSktnxRY2RK7RdZsrq4 G0r6tLEpL70U3Pn5fcn2pTFz5571nt9ql5zJHlUVv1zcCRqHp13utE7yiDEkzw== Received: from smtpclient.apple (unknown [IPv6:2a02:8084:d6bb:510:a88f:9fc6:d013:a3bf]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: melifaro/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PWr285S11z11Dg; Wed, 8 Mar 2023 11:39:28 +0000 (UTC) (envelope-from melifaro@freebsd.org) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: GSoC 2023 project ideas From: Alexander Chernikov In-Reply-To: Date: Wed, 8 Mar 2023 11:39:17 +0000 Cc: freebsd-hackers@freebsd.org, "Bjoern A. Zeeb" , mmokhi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <6A5AFC7F-DF90-4B08-A88A-7512D1C930CE@freebsd.org> References: To: obiwac X-Mailer: Apple Mail (2.3731.400.51.1.1) X-ThisMailContainsUnwantedMimeParts: N [adding bz@ and mmokhi@ to the cc] > On 8 Mar 2023, at 09:19, obiwac wrote: >=20 > Hi! Hi! >=20 > I'm a CS student from Belgium and I'm interested in participating in > Google Summer of Code as a FreeBSD contributor. I've been using > FreeBSD for a bit over two years and I've already upstreamed a few > patches over that time, but I'd like to give back in a little more of > a meaningful way :) That=E2=80=99s awesome! >=20 > More specifically, I was thinking of proposing work on the Linux KPI > for compiling network drivers on FreeBSD once the applications open > (see SummerOfCodeIdeas: > = https://wiki.freebsd.org/SummerOfCodeIdeas#Add_support_for_building_Linux-= only_network_drivers). >=20 > I see netlink was the subject of a previous SoC project by Sean Ng > (and their mentor melifaro@), so it would seem at least some part of > this has already been done. Would someone potentially be interested in > mentoring such a project (perhaps mmokhi@ or melifaro@)? Is there > maybe something you'd like to add on top of what's currently on the > ideas page wrt to this? Netlink indeed landed in FreeBSD 13/14. I=E2=80=99ll leave the reply to Bjoern as he certainly have more context = on the improvement opportunities in the LinuxKPI space. >=20 > I'm posting this message is mostly to gauge overall interest in this > idea in particular or if I'd be better off focusing my efforts on > something else... Personally I'd be very interested in working on this > if only for personal use, but maybe there's something I'd be equally > interested in which is more useful to more people ;) >=20 > Kind regards, > Aymeric >=20 /Alexander From nobody Wed Mar 8 11:50:45 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PWrJn5bttz3wJhJ for ; Wed, 8 Mar 2023 11:52:09 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "weser.webweaving.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PWrJm6ZNxz4GVZ for ; Wed, 8 Mar 2023 11:52:08 +0000 (UTC) (envelope-from dirkx@webweaving.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=webweaving.org header.s=shared header.b=TM1W1U7i; spf=pass (mx1.freebsd.org: domain of dirkx@webweaving.org designates 148.251.234.232 as permitted sender) smtp.mailfrom=dirkx@webweaving.org; dmarc=pass (policy=none) header.from=webweaving.org Received: from smtpclient.apple (217-102-131-103.cable.dynamic.v4.ziggo.nl [217.102.131.103]) (authenticated bits=0) by weser.webweaving.org (8.17.1/8.17.1) with ESMTPSA id 328Boj77046528 (version=TLSv1.2 cipher=ECDHE-ECDSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 8 Mar 2023 12:50:46 +0100 (CET) (envelope-from dirkx@webweaving.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=webweaving.org; s=shared; t=1678276246; bh=38iiiIrpMcxNuo3h/Wgq+0LyUtZRC6TsLyRaV25AKwo=; h=From:Subject:Date:To; b=TM1W1U7iGDt3zRFLLD8PGvGrwuCV5RwPZwtVqaCSjzoFs0cRzBDqJ2iTSXtFaPNRQ W03ygq5jpKj56CFr5u8I4LVUWyPzev2golEAZJpLa59fFYJUlMTmsvZvkX1kxssF9r 5ZD7M+wB8v/wRKVWn08ZQwpDozAtOr6kMGafMpig= X-Authentication-Warning: weser.webweaving.org: Host 217-102-131-103.cable.dynamic.v4.ziggo.nl [217.102.131.103] claimed to be smtpclient.apple From: Dirk-Willem van Gulik Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: ZFS and Encryption at dataset level Message-Id: <8BAC3BC0-63B2-449D-BF0E-8E5A0A42F1E0@webweaving.org> Date: Wed, 8 Mar 2023 12:50:45 +0100 To: FreeBSD Hackers X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.4 (weser.webweaving.org [148.251.234.232]); Wed, 08 Mar 2023 12:50:46 +0100 (CET) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[webweaving.org,none]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[webweaving.org:s=shared]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[webweaving.org:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:148.251.0.0/16, country:DE]; TO_DN_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_XAW(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4PWrJm6ZNxz4GVZ X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Just some words of thanks & praise -- compliments to those & all the = hard work of getting ZFS encryption at data set into 13.1. Was an absolute breeze to configure & install - and very easy to manage = ! And flexible enough to integrate with PKCS11 and HSM's. My compliments to all ! Dw= From nobody Thu Mar 9 21:16:13 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PXhnQ1S1fz3xP33 for ; Thu, 9 Mar 2023 21:16:26 +0000 (UTC) (envelope-from ashuanand29@gmail.com) Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PXhnP4GB2z3LTv for ; Thu, 9 Mar 2023 21:16:25 +0000 (UTC) (envelope-from ashuanand29@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=mR9pGdGE; spf=pass (mx1.freebsd.org: domain of ashuanand29@gmail.com designates 2607:f8b0:4864:20::b35 as permitted sender) smtp.mailfrom=ashuanand29@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yb1-xb35.google.com with SMTP id e82so3372962ybh.9 for ; Thu, 09 Mar 2023 13:16:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678396584; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=Fekk2j1pdeZRoW9bheKcFyiY+MxYx+6fhtgIacgAar0=; b=mR9pGdGEafBZRFMnxVQBlX64jNHjCohrHrTv6i9yWX/gfCj1cyfdnSm9GKBlwKmV4R 34YlhdHUkADz0TteZfw2RZNFV+2/Tp2YD3nPRjlEjXpYOaZ40x1osom5gwT8zeDyODF6 y8yr8pPpCv/ZS6FOEGhpv9DnXFHEXEyJwZHRqBsIyqQw0tw08AUhhmB42NOaBzfnlkqD aL8OK2w7YP+qWT6jTwWJm4nFnole1MEXq9KrDB65QrR8fEZQpWDjSWz6Fhfxs/7EoM+Q 1jmDQrrf65qsj/xNGKM6A56TYWfbog5LZcbvkuzT467UYRKH8HtryTRm6zf/xyVYaqlZ CBtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678396584; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Fekk2j1pdeZRoW9bheKcFyiY+MxYx+6fhtgIacgAar0=; b=jjopOb7IddPHwJvSH03yJMhsR10cMKm9vMBHJdmHtyiOObREyPjinHIix34ob7TVYZ zaU+YFYOHra7Tafi9tWaELwqibEg63nwXzZz5NKYF0eaGdPlxYtqQaWJbnzUSq8fm+hd vXQLOwdLTUIKAqQG8sBqf4yimjiE66Vk4BxJl36LGluYh2wmsaomTkHLmOovebT2t0UJ MrOV5taeLZ/aQcSoogwGFZ2CI7Ue1D5HDP+quIetItdrXtVzGJpBYlo6rxyCh6jpELze 8MMk40L3g7r5oMAo1gMinXSwyd3/c/HDNJIn1/0pm83FH+Gl9FQmyzVhHROyhq9L0srb Pwwg== X-Gm-Message-State: AO0yUKU6u2OBo4mWc1e0ooQ13nXKMc0RHUmQAr7xhH7UnPoUe0lgilJ5 Sv0zchnFGGGDWT9dxOFGaf50IF6y1nWSzt3ocz+s/kngiqx2p/WN X-Google-Smtp-Source: AK7set8DGr6dNyASCfyJKju5qIgPgrCPLAHk+F8OB3/lS6+wtxDuMJxSd4SmVQZhdnBKyhiEPtmpxWddrYeO1ygD/bQ= X-Received: by 2002:a25:8b8f:0:b0:906:307b:1449 with SMTP id j15-20020a258b8f000000b00906307b1449mr14164052ybl.5.1678396584579; Thu, 09 Mar 2023 13:16:24 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Ashutosh Anand Date: Fri, 10 Mar 2023 02:46:13 +0530 Message-ID: Subject: GSoC 2023: Implement MPLS support for FreeBSD To: "freebsd-hackers@FreeBSD.org" Content-Type: multipart/alternative; boundary="0000000000003757ce05f67e2812" X-Spamd-Result: default: False [-3.83 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; NEURAL_HAM_SHORT(-0.83)[-0.835]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_EQ_ADDR_ALL(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b35:from]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_FROM(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4PXhnP4GB2z3LTv X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N --0000000000003757ce05f67e2812 Content-Type: text/plain; charset="UTF-8" Hi! I am Ashutosh Anand, a final year undergrad at NITK, India. I am one of the prospective student for the following GSoC project. I have been personally interested in implementing this project for FreeBSD. My initial contact with the mentor (Alexander Chernikov) was successful, however, *for the past 3-4 days my communication with him has been off*. Since there were no other assigned mentors for this project I decided to write this email, *to query if I could continue researching and building my proposal for this project*. *Apologize* for the inconvenience caused. Hoping for a positive reply! Regards, Ashutosh Anand CSE student at NITK --0000000000003757ce05f67e2812 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hi!

I am Ashutosh Anand, a final year undergrad at NITK, India. I am one of the prospective student for the follo= wing GSoC project.

=C2=A0

I have been personally interested in implementing this project for FreeBSD. My initial contact with the mentor (Alexander Chernikov) was successful, however, for the past 3-4 days my communication with him has been off.

=C2=A0

Since there were no other assigned mentors for this project I decided to write this email, to query if I co= uld continue researching and building my proposal for this project. <= /span>

=C2=A0

Apologize for the inco= nvenience caused. Hoping for a positive reply!

=C2=A0

Regards,

Ashutosh Anand

C= SE student at NITK
--0000000000003757ce05f67e2812-- From nobody Fri Mar 10 11:18:35 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PY3TP0rw9z3xQJL for ; Fri, 10 Mar 2023 11:18:49 +0000 (UTC) (envelope-from ltning-freebsd-hackers@anduin.net) Received: from mail.anduin.net (mail.anduin.net [185.42.170.45]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PY3TK6fB4z3FBV for ; Fri, 10 Mar 2023 11:18:45 +0000 (UTC) (envelope-from ltning-freebsd-hackers@anduin.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=anduin.net header.s=dkim2021 header.b=TIiIc59K; spf=pass (mx1.freebsd.org: domain of ltning-freebsd-hackers@anduin.net designates 185.42.170.45 as permitted sender) smtp.mailfrom=ltning-freebsd-hackers@anduin.net; dmarc=pass (policy=reject) header.from=anduin.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=anduin.net; s=dkim2021; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From: References:To:Subject:MIME-Version:Date:Message-ID:Sender:Reply-To:Cc: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=RDvwIovWdVcN0RMV2gyNL+OaqRRdL8zbA16qdZBlB3s=; t=1678447125; x=1679311125; b=TIiIc59KZK0zXAJ6y/+h9Y/OE6qBDwf58b/V1n+f1W9PJGdfi/pX6wc6TSEeb5/3cPsm/tVJ2Qu f6RddJnCkO3Dp4SbeROxs36mzNFA0k9hZ1pqjOJiJILnBGoN9lzjZBz2ZS2eBL8Paj4+KdgBoZMaj AwKTYXWMumr66RQGBRxGy3Fw7Y19wqXl2D6oJ/H4K9c0NwI1uCKAdmer4TiiJwzZxzqwlSv2btU3G fThmC1/4VmUKLEAUhb2xY6vO8nBdvhpgtMMYB0FZ+3WhEyRdNAxE4yy4Wv/hofKY83qanqr2z5waS E9roVYriWok4DlhqVojdH/IX17SViHuP0bww==; Received: by mail.anduin.net with esmtpsa (TLS1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.95 (FreeBSD)) (envelope-from ) id 1paalg-000CVB-5R for freebsd-hackers@freebsd.org; Fri, 10 Mar 2023 11:18:37 +0000 Message-ID: Date: Fri, 10 Mar 2023 12:18:35 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: ZFS and Encryption at dataset level To: freebsd-hackers@freebsd.org References: <8BAC3BC0-63B2-449D-BF0E-8E5A0A42F1E0@webweaving.org> Content-Language: en-US From: ltning-freebsd-hackers@anduin.net In-Reply-To: <8BAC3BC0-63B2-449D-BF0E-8E5A0A42F1E0@webweaving.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SA-Authenticated: Yes X-Spam-Score: -1.9 X-Spam-Level: - X-Spam-Report: host: mail.modirum.com | contact: hostmaster@modirum.com | scores: BAYES_00=-1.9,NICE_REPLY_A=-0.001,NO_RELAYS=-0.001 | autolearn=no autolearn_force=no, score=-0.001 X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[anduin.net,reject]; R_DKIM_ALLOW(-0.20)[anduin.net:s=dkim2021]; R_SPF_ALLOW(-0.20)[+ip4:185.42.170.45/32]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:62248, ipnet:185.42.170.0/24, country:EE]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DKIM_TRACE(0.00)[anduin.net:+]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_NO_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4PY3TK6fB4z3FBV X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On 3/8/23 12:50, Dirk-Willem van Gulik wrote: > Just some words of thanks & praise -- compliments to those & all the hard work of getting ZFS encryption at data set into 13.1. > > Was an absolute breeze to configure & install - and very easy to manage ! And flexible enough to integrate with PKCS11 and HSM's. This sounds really interesting! We've done some hackery to achieve something similar, but I'd be really interested in knowing what you've done in this respect, and how! Hope it's something you can share publicly :) /Eirik > > My compliments to all ! > > Dw > From nobody Fri Mar 10 12:09:32 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PY4dk6ll9z3xSV9 for ; Fri, 10 Mar 2023 12:11:06 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "weser.webweaving.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PY4dk3Dbpz3LN6 for ; Fri, 10 Mar 2023 12:11:06 +0000 (UTC) (envelope-from dirkx@webweaving.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (217-102-131-103.cable.dynamic.v4.ziggo.nl [217.102.131.103]) (authenticated bits=0) by weser.webweaving.org (8.17.1/8.17.1) with ESMTPSA id 32AC9XmS059116 (version=TLSv1.2 cipher=ECDHE-ECDSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 10 Mar 2023 13:09:34 +0100 (CET) (envelope-from dirkx@webweaving.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=webweaving.org; s=shared; t=1678450174; bh=nRpFlcOkZH6HfBF2NB9vOsZbwk7WCiO5kPrKrVuIYAo=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=pCBuj/cOL6aMuJYNImOzBB4C2+K+MjvDLU2XnY5s0zFSl7mMssG/hRWMfgxAzTRba c1oqKL1yienuCD3SiV8yxzkGcj++tAUkjcvcg2uSWsiRR4tOYXZkYMqsHFgQqx6QOO P3alfZEU5DznM3+YU8g/IKdJR00eDjud2vVjn1Cc= X-Authentication-Warning: weser.webweaving.org: Host 217-102-131-103.cable.dynamic.v4.ziggo.nl [217.102.131.103] claimed to be smtpclient.apple Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: ZFS and Encryption at dataset level From: Dirk-Willem van Gulik In-Reply-To: Date: Fri, 10 Mar 2023 13:09:32 +0100 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <8BDBF2BE-77BB-481A-BFB0-2488A8FFA118@webweaving.org> References: <8BAC3BC0-63B2-449D-BF0E-8E5A0A42F1E0@webweaving.org> To: ltning-freebsd-hackers@anduin.net X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.4 (weser.webweaving.org [148.251.234.232]); Fri, 10 Mar 2023 13:09:34 +0100 (CET) X-Rspamd-Queue-Id: 4PY4dk3Dbpz3LN6 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:148.251.0.0/16, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > On 10 Mar 2023, at 12:18, ltning-freebsd-hackers@anduin.net wrote: >=20 > On 3/8/23 12:50, Dirk-Willem van Gulik wrote: >> Just some words of thanks & praise -- compliments to those & all the = hard work of getting ZFS encryption at data set into 13.1. >> Was an absolute breeze to configure & install - and very easy to = manage ! And flexible enough to integrate with PKCS11 and HSM's. >=20 > This sounds really interesting! We've done some hackery to achieve = something similar, but I'd be really interested in knowing what you've = done in this respect, and how! Hope it's something you can share = publicly :) It is a bit hard to share the HSM integration; as that relies on their = pkcs#11 tooling, the rather specific key references and the way their = callback for N from M works. But in short; we boot the machine without mounting the encrypted = partions. The basic assumption (with some measures like trusted boot, = etc) is that this environment is sufficiently safe (i.e. shells and = share libraries cannot easily be hijacked) - also relative to the issues = with a root empowered entity their access to kernel and the keys stored = there once you have loaded them. We keep a ZFS encryption key on the machine as a file.=20 Depending on risk of loss v.s. v.s. pain of not getting up after a = reboot v.s. risk of leak v.s. risk of ops-errors/process-errors and the = level of 4-eye needed; we then have a few different options. =20 In general, this file is then encrypted with a (second) file-key. This = file key is stored encrypted against a set of public keys. At the most = strict end - we then have an HSM decrypt this key - with a 2 of M check = on chipcard access & pin entry on a desktop device that is integrated = with the HSM remotey. At the simplest case - we just use a OpenPGP = chipcard in a reader with a pincard on a normal desktop with just GPG = and OpenSC and SSH.=20 Two of these below (one plain & direct; when you essentially trust the = user; one with the file-key separation -- which lets us rotate that file = with ease as long as there is enough 4 eye/no backup/separation-of-roles = on the encrypted key file). Below is simplified/edited a bit - we have some sudo-glue in place as = `load-keys' is under zfs; we basically made a 'zfs-load-key' to curtail = 'zfs' power. Dw=20 #!/bin/sh # # Authorized keys in ~/.ssh/authorized_keys on target # # command=3D"cat /.zfs-key; /sbin/zfs load-key -L prompt tank/enc && = zfs mount -a" ssh-rsa AAAAB.... # # Initial generate & encrypt key with: =20 # # openssl rand -base64 128 | |tr -d '\n' |\ # gpg -R xx -R xx -R .... -a -e --no-encrypt-to > /.zfs-key #=20 # Assumption: Unlocking user `trusted` as root; no 4 eyes, no role = separation, etc # set -e if [ $# =3D 0 ]; then echo Syntax: $0 hostname ... exit 1 fi if ! /usr/local/bin/gpg --card-status 2> /dev/null; then echo Seat your personal/npi chipcard first exit 2 fi FIFO=3D/tmp/fifo.$$ KEYID=3D${KEYID:-XXXXXXXXXX} /usr/bin/mkfifo $FIFO E=3D0 ( for host in $* do cat $FIFO |\ /usr/bin/ssh -F /dev/null -i $SSHKEY $SSH_EXTRA_ARGS -T = -l root "$host" |\ /usr/local/bin/gpg --quiet --default-key "$KEYID" >> = $FIFO done ) || E=3D$? rm -f $FIFO exit $E #!/bin/sh # # Authorized keys in ~/.ssh/authorized_keys on target # # command=3D"/usr/local/sbin/zfs-hsm-load-key && zfs mount -a" ssh-rsa = AAAAB.... # # zfs-hsm-load-key essentially does # get /.zfs-key-to-keyfile decrypted via the hardware # use it to decrypt /.zfs-key # sudo zfs load-key the output # remount and restart jails as needed. # # Initial generate & encrypt key with: =20 # # openssl rand -base64 128 | |tr -d '\n' |\ # hsm-file -enc -k - > /.zfs-key-to-keyfile # # openssl rand -base64 128 | |tr -d '\n' |\ # hsm-file -enc -k - -f /.zfs-key-to-keyfile -k /.zfs-key #=20 set -e if [ $# !=3D 1 ]; then echo Syntax: $0 hostname ... exit 1 fi if ! hsmlnk -p 9731 -lrp; then echo Make sure the extender connected to USB exit 1 fi ssh -L 9731:localhost:9731 -F /dev/null -i $SSHKEY $SSH_EXTRA_ARGS -T -l = hunclk $host exit $E From nobody Fri Mar 10 21:27:05 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PYJzN73NDz3xxlm for ; Fri, 10 Mar 2023 21:27:12 +0000 (UTC) (envelope-from seafork@disroot.org) Received: from knopi.disroot.org (knopi.disroot.org [178.21.23.139]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PYJzN03vPz4R6b for ; Fri, 10 Mar 2023 21:27:11 +0000 (UTC) (envelope-from seafork@disroot.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=disroot.org header.s=mail header.b=Rq51MkBt; spf=pass (mx1.freebsd.org: domain of seafork@disroot.org designates 178.21.23.139 as permitted sender) smtp.mailfrom=seafork@disroot.org; dmarc=pass (policy=reject) header.from=disroot.org Received: from localhost (localhost [127.0.0.1]) by disroot.org (Postfix) with ESMTP id F207144540 for ; Fri, 10 Mar 2023 22:27:09 +0100 (CET) X-Virus-Scanned: SPAM Filter at disroot.org Received: from knopi.disroot.org ([127.0.0.1]) by localhost (disroot.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mFZLr9hfe3iR for ; Fri, 10 Mar 2023 22:27:09 +0100 (CET) To: freebsd-hackers@freebsd.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=disroot.org; s=mail; t=1678483629; bh=jhsyWHhwBZEcaNJLq2ZZxwXNfC/gkmmaLKjcyz3F004=; h=To:From:Subject:Date; b=Rq51MkBtpyiAkYDVbxt1UtKIzfOEGDuu4iTyAoGz3WHsSeEs/uzVMkVKzPdMM2YXh ibV5Bq5yhTcD8ClQkwFVcPwSfevqhB6zGZZcc4saMLC6GYbXG55Z83B0o1tVqDS+lV d6CAr1sX+d8Q+bGsSztSrgFPdOPKL4i3EQzZ+jtdQBwSBbQ12KrJVHjgukvtQhupcR IbVikyFf4ce/dEoS2HwtfXCW46Plcok+c88+1NxPIqbAG56ZFv7uFHF8pzPY3DgmgJ 0MKENZEsDPchJRK25SC7jOj7t2mG8ks9dUrStVCoc5qPg91sAtpCs9LuDVhFOVqjHh tGbCfGb3/BTrQ== From: Lucy Marsh Subject: Adding the secure_getenv call to FreeBSD's libc Message-ID: <64fc1989-aa35-7a5f-fc0a-bc649b68ecee@disroot.org> Date: Fri, 10 Mar 2023 16:27:05 -0500 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[disroot.org,reject]; R_SPF_ALLOW(-0.20)[+a]; R_DKIM_ALLOW(-0.20)[disroot.org:s=mail]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[disroot.org:+]; ARC_NA(0.00)[]; ASN(0.00)[asn:50673, ipnet:178.21.23.0/24, country:NL]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4PYJzN03vPz4R6b X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Dear FreeBSD Hackers, I was wondering if adding the glibc extension call, `secure_getenv`, to FreeBSD's libc is allowed. Obviously, this would not only need to be permitted but also wanted. In that latter department, I could see the need arise for `secure_getenv` when porting applications written for Linux as they are often written targeting glibc. Also, this addition would bring us more inline with other libc implementations such as musl libc. Thank you for your time, Lucy From nobody Fri Mar 10 21:39:22 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PYKFZ1cx3z3xyJj for ; Fri, 10 Mar 2023 21:39:30 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PYKFY6xYNz4SJc for ; Fri, 10 Mar 2023 21:39:29 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Authentication-Results: mx1.freebsd.org; none Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id A64213C0199; Fri, 10 Mar 2023 21:39:22 +0000 (UTC) Date: Fri, 10 Mar 2023 21:39:22 +0000 From: Brooks Davis To: Lucy Marsh Cc: freebsd-hackers@freebsd.org Subject: Re: Adding the secure_getenv call to FreeBSD's libc Message-ID: References: <64fc1989-aa35-7a5f-fc0a-bc649b68ecee@disroot.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <64fc1989-aa35-7a5f-fc0a-bc649b68ecee@disroot.org> X-Rspamd-Queue-Id: 4PYKFY6xYNz4SJc X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Mar 10, 2023 at 04:27:05PM -0500, Lucy Marsh wrote: > Dear FreeBSD Hackers, >=20 > I was wondering if adding the glibc extension call, `secure_getenv`, to= =20 > FreeBSD's libc is allowed. Obviously, this would not only need to be=20 > permitted but also wanted. In that latter department, I could see the=20 > need arise for `secure_getenv` when porting applications written for=20 > Linux as they are often written targeting glibc. Also, this addition=20 > would bring us more inline with other libc implementations such as musl= =20 > libc. Looking at the musl implementation, it looks like this is part of a set of environment (mostly path) hardening changes in libc. On the whole they seem like reasonable things to do if we haven't already done them on an adhoc basis. -- Brooks From nobody Sat Mar 11 15:43:20 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PYnJD4t6Sz3xmX8 for ; Sat, 11 Mar 2023 15:43:24 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PYnJC2LxGz3KND for ; Sat, 11 Mar 2023 15:43:23 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=eLbiryB8; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=RlBxJIHg; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.29 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id BD1AC5C009F for ; Sat, 11 Mar 2023 10:43:22 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Sat, 11 Mar 2023 10:43:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm2; t=1678549402; x=1678635802; bh=ujhMyDiESf9TX6sJKAoa8tQtI 0nbLJXIkqScIWbxK+0=; b=eLbiryB8qY9wjB8q5T/LVYCB7uTKz8X4MrI2oMVhd Zg8XhD8TyQXIHGJGoTX0zQzrqZq+m2JVuYc9wa8Fyuh6NspWGDMPMF4lwbnPpJR3 N0TztwN7Lo0vmPkd3u6buep9RI+kvdTO6PVTZbUy6Sip5QrNpTP/0Z4kgQhb8xOJ B+HJkUF/+D/AgQZf+Dv7Quf52e9Q2WONBGxNb2roCqbhwaq2MtOZidskTlbVMBWi xlYzsg2q+GY35Xr2J2NXEA6TmPZAhHqlShIZtqM5RfTLalc6cEPtssfPcWuw6M5X uf+PahDpQzSyxPndReUz7JcyEQwEEwH5ByaOTRPFlHWKA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1678549402; x=1678635802; bh=ujhMyDiESf9TX6sJKAoa8tQtI0nbLJXIkqS cIWbxK+0=; b=RlBxJIHgc1YVPh0CQPjpYAiODQB+4iDX0Ptho38oWQTgkyDoQNK kHMgJpd+TU4HJiZLCwebvKBER9B8Wjmgm51NF2j/ygyIIYBXj6Ub4842rGLpxMHm id4qOXpaiLhYZ7MKMxL/TxuUHgak2/Ujv1pCDGVKhOWk+jbGtYW24sOJuMz1EWUC QnHPiafmvhj9iEa25g4pP1j3Cv8KFvLK6kEbOkdP1Kn4rxU5htTgsqUlK07PIBB0 DtQcpRl/4GMpp7PF0ILgDQqWyN60jBrxTxKphtaTUkRxH5ZMzHKDEuO6duiKBAem GOE24kvaJbdsUNn+YY2KPU9ziGA8y4t6hgw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvddvtddgheegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtugesthdtredttd dtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgrthht vghrnhepveduffeivdfffffghfegfeejfefftdeiteehteekfefhvdefgfettdeuheegff eunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepvhho ihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 11 Mar 2023 10:43:22 -0500 (EST) Date: Sat, 11 Mar 2023 15:43:20 +0000 From: void To: freebsd-hackers@freebsd.org Subject: quirks Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline X-Spamd-Result: default: False [-2.73 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.997]; NEURAL_SPAM_SHORT(0.87)[0.870]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_FROM(0.00)[f-m.fm]; TO_DN_NONE(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4PYnJC2LxGz3KND X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Hello hackers@ What are 'quirks', how are they applied, why, where can I read up about them, are they documented? I have a failing disk, this gets displayed in the console: kernel: umass2: SCSI over Bulk-Only; quirks = 0xc105 tia, -- From nobody Sat Mar 11 16:45:44 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PYphM1wN0z3xq7B for ; Sat, 11 Mar 2023 16:45:55 +0000 (UTC) (envelope-from mail@souji-thenria.net) Received: from alisa.souji-thenria.net (alisa.souji-thenria.net [188.68.37.165]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PYphK5CJnz3hjx for ; Sat, 11 Mar 2023 16:45:53 +0000 (UTC) (envelope-from mail@souji-thenria.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=souji-thenria.net header.s=20220813rsa header.b=0FYe6zAm; spf=pass (mx1.freebsd.org: domain of mail@souji-thenria.net designates 188.68.37.165 as permitted sender) smtp.mailfrom=mail@souji-thenria.net; dmarc=pass (policy=none) header.from=souji-thenria.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=souji-thenria.net; s=20220813rsa; t=1678553144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p6Ch8NB1SN8aTFMVilkwPBK28ZGIsCmHwPIKrvPLweI=; b=0FYe6zAm0DroQXVVoj7lUu2+pXyhehocqnxdiwlLoUDi4NdKFeBjAoe7w0C1MnoA1Ndhux 1dKygQAFJNIlvvwoO1T4rZCUKuK1HA3iyj8LPiFw0IQaQcPWvTavYbuaZaBbVjMk4LnlWR PLNC5fcpIku6lAWR6YjJnIcyq95cGoakoK79K8goCuVaxeqdjiV1LWLLcNi8ToQjOEm7e2 AWeRAxEGIOQ48/nXzTcq9Sj4B8YC/JNixav6a1JCb1cGEdYL+gj7gQUCDW3dzAlQzZoraP B5yLATQJBd0GAN9iTKU89h9f4q+z014FId/4IhSsCcEnSBdr/nUeca+fRjRn9A== Received: from [192.168.178.41] (nat-178-19-229-24.net.encoline.de [178.19.229.24]) by alisa.souji-thenria.net (OpenSMTPD) with ESMTPSA id 8072ffea (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Sat, 11 Mar 2023 17:45:44 +0100 (CET) Message-ID: Date: Sat, 11 Mar 2023 17:45:44 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: quirks To: freebsd-hackers@freebsd.org References: Content-Language: en-US From: Souji Thenria In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[souji-thenria.net,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[souji-thenria.net:s=20220813rsa]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[souji-thenria.net:+]; ASN(0.00)[asn:197540, ipnet:188.68.32.0/20, country:DE]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4PYphK5CJnz3hjx X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On 3/11/23 16:43, void wrote: > What are 'quirks', how are they applied, why, where can I read up about > them, are they documented? Hey tia, I have found this [http://www.root.org/~nate/freebsd/scsi/quirks.html]: FreeBSD drivers make every attempt possible to support the standards behind hardware. Where possible and not in conflict with the standard, they also attempt to work around hardware which doesn't strictly conform. However, some devices have flaws which can't be worked around while keeping the driver compatible with the standard. For these devices, we have created a quirks mechanism to indicate to the driver that it must avoid certain commands or use them differently with a specific model and/or version of hardware. This document focuses on identifying and committing quirks for storage hardware involving CAM and UMASS but is applicable to other areas. Based on that, quirks are just identifiers for the driver, to compensate for non-standard hardware implementations. -- Souji Thenria