From nobody Mon Jul 5 15:09:18 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 45A6F11D77F3; Mon, 5 Jul 2021 15:09:33 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJTcX1fq8z4RxW; Mon, 5 Jul 2021 15:09:32 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f182.google.com with SMTP id e13so1222868ilc.1; Mon, 05 Jul 2021 08:09:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=RUNYyPwAWHPMEvH6m9X36t5VSUOWMvaV5YQEAlVqRrY=; b=dQ+m73yuc+BXFvfJGahE/8y9pcTZRA0VsDiMo0nNMp8rmGn57ADf4EXM9pKEO74nSp sR2wC4Sh4yRtRdbButsBuSou7hHnT6xUDqa2sh53iq+8pZFckApsS0vw4B9zIi5f457o d7NJX0HD1EtjzgYw+gsyh6u3yUdQI/VXsMPBCiKG4uUmM7ePdR1SGM1HrPDP9+1Fjb7Z tILGKu81gZfSLaS14RMoDRqs5OsvDOKogZkqbGdxvQyjFgJNGUrdmpPElgcZXnCdJJY/ BnCn0ji/RJwR8NjOBkykEStI/zjyDkHIIOR6BnD1Dl+rkEou/vZ30Z3rElyzR9yZ16YG AOCQ== X-Gm-Message-State: AOAM533rTnUdet/hIzPNH3PElvG996IZT/+EIFRrOwW1H9PCXcE4PHuO sXPwrC30QsmuL4Va7kJWMLZVNxNEN9RveM+r3xZV58y0 X-Google-Smtp-Source: ABdhPJzk9FiLnMr4gyZ4/EqetI1k4pKHvc9kg2VdiAkpkbi9CAFv/Zt3NBXGCb0uHxlUQiqrYx0+Gh7K0VQuJTnHZ4Q= X-Received: by 2002:a92:d0d1:: with SMTP id y17mr11372496ila.256.1625497770284; Mon, 05 Jul 2021 08:09:30 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Ed Maste Date: Mon, 5 Jul 2021 11:09:18 -0400 Message-ID: Subject: Migrating to LLVM binutils tools (ar, nm, addr2line, etc.) To: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers , Dimitry Andric , Alexander Richardson Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GJTcX1fq8z4RxW X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.182 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-0.98 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.166.182:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.97)[-0.974]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_SPAM_MEDIUM(0.99)[0.992]; SPAMHAUS_ZRD(0.00)[209.85.166.182:from:127.0.2.255]; R_DKIM_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.182:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.182:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-toolchain,freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N FreeBSD migrated from GNU binutils to versions from ELF Tool Chain, starting in 2014. At that time there were no usable LLVM versions of those tools, but they have been developing rapidly since then. Now I think it may be prudent to migrate to the LLVM tools where they exist, for both functionality and maintainability reasons. I'd like to allow use of link-time optimization (LTO) in the FreeBSD base system. LTO runs optimization passes over the entire executable (or library) at link time and thus allows for more effective optimization than when performed on individual compilation units. When using LTO object files (.o) including those contained in static library archives (.a) contain LLVM IR bitcode rather than target object code. This means that utilities that operate on object files need to support LLVM IR; we currently use a number of bespoke tools and ones obtained from ELF Tool Chain that do not have this support. Alex Richardson also pointed out that asan (address sanitizer) produces a useful backtrace only if addr2line is llvm-symbolizer. Like ELF Tool Chain the LLVM tools aim for command line and output format compatibility with GNU binutils, although there are a few minor differences. Where these cause a material issue (breaking a port or eliminating required functionality) we can submit LLVM bugs and work on patches. In the past we provided build knobs to control individual utilities (e.g. WITH_LLD_IS_LD); I'd like to avoid perpetuating that here. It seems individual knobs (WITH_LLVM_AR_IS_AR, WITH_LLVM_NM_IS_NM, WITH_LLVM_SYMBOLIZER_IS_ADDR2LINE etc.) will introduce extra complexity without adding much value. Alex is working on a patch now and will follow up shortly, but I wanted to email the list as a heads-up, and see if there are any comments or concerns. Potential next steps are: - Introduce new build knob - Iterate on exp-runs and call for testing - Switch to LLVM tools by default - Major release (14.0) - Retire knob, leaving only the LLVM implementation. From nobody Mon Jul 5 16:32:00 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5B8287CE92D for ; Mon, 5 Jul 2021 16:32:03 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com [IPv6:2a00:1450:4864:20::229]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJWRk5l8kz4j94; Mon, 5 Jul 2021 16:32:02 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lj1-x229.google.com with SMTP id w11so25438638ljh.0; Mon, 05 Jul 2021 09:32:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=wwCo42yd/6YII45ZPiAxfcMuQXFGwkJzgHoyUADHtrQ=; b=eKvIqFoMZSdBuI5gftLX3C9Dhj89/JeKJRxGBr63Wo91G3aCD6sR8I9ZgPuVirCAB2 wJGL2FNPkJS3tLSqMK9ab16zSLxXYP6g4eML4Sd+UbS/tgXbfv6cJNE+/XSFi7wAO5oG YTs6IT+6jqz9z/3ltb2QGPdKKICPROczPTC0RTsMV0bzcfrqQK3eRd0uVRIHee7L3Wjf SJrYZ2gcjdM8Ulv1ivZVb7rOc95CmIBG2w+Wkk4ChzyvQYsaMBtOqaGkZl39UgR4simf YyHlrnHApmMlgnzRuFscF3L2tlSyrch5Fy7zPjUBA6JvMVQUfzPTkeXk2h0WWVC+vq4H wzVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=wwCo42yd/6YII45ZPiAxfcMuQXFGwkJzgHoyUADHtrQ=; b=kaH5EkOvydleluOxgx4TxNKmP2QWfjWE7Pdh1y64hFp+vg6GCqEWeVe6eFIj0HPaqq 3o86EajE4+qVU61bwQdVsHl6PFr3lNeLyqIpp3aA9zDt1Oy0FjkXmOmZ9CWQI0LnE+zd eySvRkrmgQyrZRzyQGOs8A7Z0kRgwMHZ8XWZ+gtWXxweIfaf6W4Uj9C9L5r5y3OJ9zAo 9/0Fz2uHwN1QUt3Y30c9ni9yAd7nwpBPNfcdnMVwSdlk8QLen6vwLJvTWF4Mm6PXWulC vtbl/vR3I/16yMVDT+EsPVtFtUGVmsllBwNyXQ93SFbdVb1ja7c1XeCKWGSOsOuh1Goy z4ug== X-Gm-Message-State: AOAM531eP5pp3SbxKHb1kK6VsJSnTz2g5ZG7663bjycxpnOEzj2PqeOQ Aarylg5W0nVVeGX9lXCFT6Q= X-Google-Smtp-Source: ABdhPJwRuGk4/hK2kPhiJDfty9mCPfZGCR3cE7g2ZlxWVnFnlCiRyk5ID4mUIky3gDhJAXCp9WGldQ== X-Received: by 2002:a2e:a4ae:: with SMTP id g14mr11954355ljm.354.1625502721258; Mon, 05 Jul 2021 09:32:01 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id c13sm1413254ljf.47.2021.07.05.09.32.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Jul 2021 09:32:00 -0700 (PDT) From: Vitaliy Gusev Message-Id: <2390FA9B-319E-45D4-BEA7-10878E43AD4B@gmail.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_283679B0-317F-482E-9F4E-E3C4F0D00D24" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Date: Mon, 5 Jul 2021 19:32:00 +0300 In-Reply-To: Cc: freebsd-hackers@freebsd.org, gljennjohn@gmail.com, Mark Johnston To: kostikbel@gmail.com References: <20210703065420.6dbafb5f@ernst.home> X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GJWRk5l8kz4j94 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=eKvIqFoM; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gusevvitaliy@gmail.com designates 2a00:1450:4864:20::229 as permitted sender) smtp.mailfrom=gusevvitaliy@gmail.com X-Spamd-Result: default: False [-1.78 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::229:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[188.187.60.230:received]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.28)[-0.277]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::229:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::229:from]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_283679B0-317F-482E-9F4E-E3C4F0D00D24 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, > > Does it mean madvise() doesn't work well in FreeBSD or test does = something wrong? >=20 > Your program does not exactly what you described above. There is a = generic > race to consume memory, and some specific details about madvise(2) on = FreeBSD. >=20 > >=46rom the code, you do: > - mmap anonymous private region > - fork > - both child and parent start touching the mmaped region. >=20 > Two processes race to consume 1/2 of RAM on your system. If one of > them happen to execute faster then another, you do get to the case = where > one of them does madvise(). But it could be that processes execute in > lockstep, and try to eat all the memory before going to madvise(). > Did you excluded this case? I believe I did all things right. You can see sleeps that serialise = execution. To check again I modified test and added time printing and = use MADV_DONTNEED: Here is source http://cpp.sh/2rd4f I=E2=80=99ve run:=20 $ ./mmapfork 2300 mmap 0x801000000 pid 40628 end 0x890c00000 len 0x8fc00000 pid 40628 pid 40629 40629: [1625500831] touch 40629: [1625500832] sleep before madvise 40629: [1625500833] madvise 40629: [1625500834] Press enter to exit 40628: [1625500845] touch 40628: [1625500846] sleep before madvise 40628: [1625500851] madvise 40628: [1625500852] Press enter to exit And you can see that child started running in 11 seconds after parent = had already called madvise() for all scope of touched memory. And finally in dmesg: pid 40629 (mmapfork), jid 0, uid 1001, was killed: out of swap space So the same result as I wrote in the first email. > Now, about the specific of madvise(MADV_FREE) on FreeBSD. Due to the = way > CoW is implemented with the shadow chain of objects, we cannot drop = the > top of the shadow chain, otherwise instead of returning zeroed pages = next > time, we would return content back in the time. It was relatively = recent > discovery, see bf5661f4a1af6931ec4b6, PR 240061. >=20 Thanks, I will look at it. > To explain it in simplified form, when there is potential old content > under the CoW copy for the mapping, we cannot drop CoW-ed pages. This > is the motivation why madvise(MADV_FREE) does nothing for your = program. > When you run two instances without fork, there is no previous content > and no Cow, so madvise() can safely remove the pages from the object, > and on the next access they are zero-filled. Do I understand right, that it should work with MADV_DONTNEED? But = =E2=80=9Cdontneed" variant doesn=E2=80=99t work.=20 >=20 > You can read more details in the referenced commit, as well as some = musings > about way to make it somewhat better. >=20 > I must say, that trying to allocated 1/2 + 1/2 of RAM this way, on a = system > without swap, is the way to ask for troubles anyway. I=E2=80=99ve just notify that other operation systems work well with = that, whereas FreeBSD has troubles. Probably something in madvise() is = not finished ? ---- Vitaliy Gusev --Apple-Mail=_283679B0-317F-482E-9F4E-E3C4F0D00D24-- From nobody Mon Jul 5 16:54:00 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 19AAF8D3861 for ; Mon, 5 Jul 2021 16:54:04 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJWx66NXxz4pD3; Mon, 5 Jul 2021 16:54:02 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lf1-x135.google.com with SMTP id t17so33584089lfq.0; Mon, 05 Jul 2021 09:54:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=7RbS79zOh28NSP6NtNY3J+45gcb8l2A9weTYu8bw+XE=; b=CD2ETdUmwwwgwtYBd7iFjO2q8QHTJNcgEnqNfrzkkbeKw3B3WpnJ0sb0yfhljaCQnz UHfJvTh8YiftLZUmYeI39hrZIxr3iLu/xbksckPzbYQs2zNi2kjGWAs/7kEQjLoO/Je+ 2ls1Jx0K9R+yLqoPhZQe/SBc3y6Y0el4t998XZqY4J3ExEd9ZGJihqeqcc8O8C4TFeBT u+3nNQSdv0IR7kbZKcR+OCo2u4WafO2CNscZ0xsNBXY1N9Nn+YSmyiB2s1AnvPLlrb3P vWDoGGMsacjVJbZdSC7/HlZrRW2J5RKTXUAD/7+7iiYJfRv8QUle+tlWCRLOamZ9jPY1 5IKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=7RbS79zOh28NSP6NtNY3J+45gcb8l2A9weTYu8bw+XE=; b=jM5iG41/XGDX4Bo75rSm7hI5zLWEcyq1/eGjw9tPwzjINwnRxfRsT9uWG9WFAjxr/u lAawefRoUBX7Z9A1Eal/6q8NxA8YprldGq9DtAgfQLzKu/7LWEjkWA5KdvEB0ZL3OAZN 78IhLyxwHng9zFJjvtnN9DYkxZusEtv8+y6bXL4FohUEDmF5i2I7P4fLdD+VCxT1g8S/ 3Oe6TVQDg/BPZlkHwt06pl7OptfMmwX1H20u9Hb9J6I0sd0KgxJfhBRlfzfRON0k3cBB +AkMZgXVGwk1GTdDVgVra8fw/tTLyawdyrh1vZSVSJEz/1lShPdiSadIowYacTAyebrb duCg== X-Gm-Message-State: AOAM533mYu5xnN5mYEvUV3zMhCS9jPEu3i+Vu8iVqYZew21A/z2yU/+B PsL7fPWr0ifzZypAcFS1U3E= X-Google-Smtp-Source: ABdhPJwKsBGVJOFVpXOs0usbR3EspZ2kBbl1IXsfBukEWgnP4Ki4Vf07t2D9fDIJgpbSG0PHEkEFOQ== X-Received: by 2002:a19:750a:: with SMTP id y10mr7030105lfe.383.1625504041379; Mon, 05 Jul 2021 09:54:01 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id x17sm1133116lfn.187.2021.07.05.09.54.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Jul 2021 09:54:01 -0700 (PDT) From: Vitaliy Gusev Message-Id: <57BCE463-6200-4F83-A321-2F0444E7F063@gmail.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_3BEFE3ED-1F3C-4A6B-AAC6-021F01426D22" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Date: Mon, 5 Jul 2021 19:54:00 +0300 In-Reply-To: Cc: freebsd-hackers@freebsd.org, Mark Johnston To: Konstantin Belousov References: <0A95973D-254A-4574-8DC7-9F515F60B873@gmail.com> X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GJWx66NXxz4pD3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=CD2ETdUm; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gusevvitaliy@gmail.com designates 2a00:1450:4864:20::135 as permitted sender) smtp.mailfrom=gusevvitaliy@gmail.com X-Spamd-Result: default: False [-2.25 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::135:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[188.187.60.230:received]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.75)[-0.753]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::135:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::135:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_3BEFE3ED-1F3C-4A6B-AAC6-021F01426D22 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, > On 3 Jul 2021, at 14:35, Konstantin Belousov = wrote: >=20 > On Sat, Jul 03, 2021 at 02:32:01AM +0300, Vitaliy Gusev wrote: >> ... >> Does it mean madvise() doesn't work well in FreeBSD or test does = something wrong? >=20 > Your program does not exactly what you described above. There is a = generic > race to consume memory, and some specific details about madvise(2) on = FreeBSD. >=20 > =46rom the code, you do: > - mmap anonymous private region > - fork > - both child and parent start touching the mmaped region. Their execution should be serialised by sleeps. Yes it is not fully = fair, but for testing purpose is enough. > Two processes race to consume 1/2 of RAM on your system. If one of > them happen to execute faster then another, you do get to the case = where > one of them does madvise(). But it could be that processes execute in > lockstep, and try to eat all the memory before going to madvise(). > Did you excluded this case? >=20 I believe I did all things right. You can see sleeps that serialise = execution. To check again I modified test and added time printing and = MADV_DONTNEED: Here is source http://cpp.sh/2rd4f and I put it = at the end of this email. I=E2=80=99ve run:=20 $ ./mmapfork 2300 mmap 0x801000000 pid 40628 end 0x890c00000 len 0x8fc00000 pid 40628 pid 40629 40629: [1625500831] touch 40629: [1625500832] sleep before madvise 40629: [1625500833] madvise 40629: [1625500834] Press enter to exit 40628: [1625500845] touch 40628: [1625500846] sleep before madvise 40628: [1625500851] madvise 40628: [1625500852] Press enter to exit And you can see that child (40628) started running in 11 seconds after = parent had already called madvise() for all scope of touched memory. And finally in dmesg: pid 40629 (mmapfork), jid 0, uid 1001, was killed: out of swap space So the same result as I wrote in the first email. > Now, about the specific of madvise(MADV_FREE) on FreeBSD. Due to the = way > CoW is implemented with the shadow chain of objects, we cannot drop = the > top of the shadow chain, otherwise instead of returning zeroed pages = next > time, we would return content back in the time. It was relatively = recent > discovery, see bf5661f4a1af6931ec4b6, PR 240061. >=20 Thanks, I will look at it. > To explain it in simplified form, when there is potential old content > under the CoW copy for the mapping, we cannot drop CoW-ed pages. This > is the motivation why madvise(MADV_FREE) does nothing for your = program. > When you run two instances without fork, there is no previous content > and no Cow, so madvise() can safely remove the pages from the object, > and on the next access they are zero-filled. >=20 Do I understand right, that it should work with MADV_DONTNEED? But = =E2=80=9Cdontneed" variant doesn=E2=80=99t work.=20 > You can read more details in the referenced commit, as well as some = musings > about way to make it somewhat better. >=20 > I must say, that trying to allocated 1/2 + 1/2 of RAM this way, on a = system > without swap, is the way to ask for troubles anyway. I=E2=80=99ve just notify that other operation systems work well with = that, whereas FreeBSD has troubles. Probably something in madvise() has not been finished yet? =E2=80=94=E2=80=94 #include #include #include #include #include #include #include int main(int argc, char *argv[]) { size_t len =3D (size_t)(argc > 1 ? atoi(argv[1]) : 1024) * 1024 = * 1024; uint8_t *ptr, *end, *p; unsigned pagesz =3D 1<<12; int pid; ptr =3D (uint8_t *)mmap(NULL, len, PROT_WRITE | PROT_READ, = MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); if (ptr =3D=3D MAP_FAILED) err(1, "cannot mmap"); end =3D ptr + len; printf("mmap %p pid %d\n", ptr, getpid()); printf("end %p len %#lx\n", end, len); fflush(stdout); pid =3D fork(); if (pid < 0) err(1, "cannot fork"); printf("pid %d\n", getpid()); sleep(pid =3D=3D 0 ? 1 : 15); printf("%d: [%ld] touch\n", getpid(), time(NULL)); p =3D ptr; while (p < end) { *p =3D 1; p +=3D pagesz; } printf("%d: [%ld] sleep before madvise\n", getpid(), = time(NULL)); sleep(pid =3D=3D 0 ? 1 : 5); printf("%d: [%ld] madvise\n", getpid(), time(NULL)); p =3D ptr; while (p < end) { int error; error =3D madvise(p, pagesz, MADV_DONTNEED); if (error) { err(1, "cannot madvise"); } p +=3D pagesz; } printf("%d: [%ld] Press enter to exit\n", getpid(), time(NULL)); getchar(); } --Apple-Mail=_3BEFE3ED-1F3C-4A6B-AAC6-021F01426D22-- From nobody Mon Jul 5 17:56:10 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9D36911D5A8D; Mon, 5 Jul 2021 17:56:23 +0000 (UTC) (envelope-from arichardson.kde@gmail.com) Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJYK33qkjz3FLm; Mon, 5 Jul 2021 17:56:23 +0000 (UTC) (envelope-from arichardson.kde@gmail.com) Received: by mail-ej1-f41.google.com with SMTP id bg14so30018650ejb.9; Mon, 05 Jul 2021 10:56:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tyeG0+uP3jlC9MrCLlel3VHagTU1hREcZfOK3elRXwA=; b=HVJOE24O28XoxatAi8OjQqBAvL1GerMyE7COahU7twRdAzSaQDzuQaZOIUYbT9qC7j G7pC5rsngGuwhVs3KRIhLEMvrxNZmEcFA/HzUnxMKKAc/IabsY9J18eL7bb0cxDa+G6H UGTiRX8jmYPeGGa5JKQNFxNX+QD894ChkPn4Jtl1NN14qeXFnbml2iJ96pMMjudpPy3C qQ393EAifztOD4BPBkErjoEZurkD/MPeUSk1+3sEPIdBRCYl4EE+1qZV6BhAbhxlDnwB PbVXeZi4EUV0thPOkE828SS5LCCNfYDeHi6ArwmewR3hCFu+gZwhWJJqg3u+PdETClcN 0cEA== X-Gm-Message-State: AOAM530EBTzagdz4t8nQ5NYVx/F3WSyC5jA3tTRmx+8Lyq296eOql+g8 QpYZrnvV6ex/bFAjxWKb8TJ2BWUNZUSQ8g== X-Google-Smtp-Source: ABdhPJxjrQM1Awz3AepQlQPCbVXaP+457AYm9RZXGKdeYP0JvbHagISHQR63OBtTYfIDFoOLFc6wEw== X-Received: by 2002:a17:906:b190:: with SMTP id w16mr14631217ejy.332.1625507782113; Mon, 05 Jul 2021 10:56:22 -0700 (PDT) Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com. [209.85.128.52]) by smtp.gmail.com with ESMTPSA id f9sm4101537edw.88.2021.07.05.10.56.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Jul 2021 10:56:21 -0700 (PDT) Received: by mail-wm1-f52.google.com with SMTP id j16-20020a05600c1c10b0290204b096b0caso493949wms.1; Mon, 05 Jul 2021 10:56:21 -0700 (PDT) X-Received: by 2002:a1c:9d8f:: with SMTP id g137mr282085wme.13.1625507781650; Mon, 05 Jul 2021 10:56:21 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alexander Richardson Date: Mon, 5 Jul 2021 18:56:10 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Migrating to LLVM binutils tools (ar, nm, addr2line, etc.) To: Ed Maste Cc: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers , Dimitry Andric Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GJYK33qkjz3FLm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, 5 Jul 2021 at 16:09, Ed Maste wrote: > > FreeBSD migrated from GNU binutils to versions from ELF Tool Chain, > starting in 2014. At that time there were no usable LLVM versions of > those tools, but they have been developing rapidly since then. Now I > think it may be prudent to migrate to the LLVM tools where they exist, > for both functionality and maintainability reasons. > > I'd like to allow use of link-time optimization (LTO) in the FreeBSD > base system. LTO runs optimization passes over the entire executable > (or library) at link time and thus allows for more effective > optimization than when performed on individual compilation units. > > When using LTO object files (.o) including those contained in static > library archives (.a) contain LLVM IR bitcode rather than target > object code. This means that utilities that operate on object files > need to support LLVM IR; we currently use a number of bespoke tools > and ones obtained from ELF Tool Chain that do not have this support. > > Alex Richardson also pointed out that asan (address sanitizer) > produces a useful backtrace only if addr2line is llvm-symbolizer. > > Like ELF Tool Chain the LLVM tools aim for command line and output > format compatibility with GNU binutils, although there are a few minor > differences. Where these cause a material issue (breaking a port or > eliminating required functionality) we can submit LLVM bugs and work > on patches. > > In the past we provided build knobs to control individual utilities > (e.g. WITH_LLD_IS_LD); I'd like to avoid perpetuating that here. It > seems individual knobs (WITH_LLVM_AR_IS_AR, WITH_LLVM_NM_IS_NM, > WITH_LLVM_SYMBOLIZER_IS_ADDR2LINE etc.) will introduce extra > complexity without adding much value. > > Alex is working on a patch now and will follow up shortly, but I > wanted to email the list as a heads-up, and see if there are any > comments or concerns. Thanks for writing this up, I've posted the initial patch as https://reviews.freebsd.org/D31060. > > Potential next steps are: > - Introduce new build knob > - Iterate on exp-runs and call for testing > - Switch to LLVM tools by default > - Major release (14.0) > - Retire knob, leaving only the LLVM implementation. From nobody Mon Jul 5 18:54:58 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CEAF511DD793 for ; Mon, 5 Jul 2021 18:55:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJZcr2ltHz3Lsh; Mon, 5 Jul 2021 18:55:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 165IswBS067636 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 5 Jul 2021 21:55:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 165IswBS067636 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 165Isw9Z067635; Mon, 5 Jul 2021 21:54:58 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 5 Jul 2021 21:54:58 +0300 From: Konstantin Belousov To: Vitaliy Gusev Cc: freebsd-hackers@freebsd.org, gljennjohn@gmail.com, Mark Johnston Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Message-ID: References: <20210703065420.6dbafb5f@ernst.home> <2390FA9B-319E-45D4-BEA7-10878E43AD4B@gmail.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2390FA9B-319E-45D4-BEA7-10878E43AD4B@gmail.com> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GJZcr2ltHz3Lsh X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N On Mon, Jul 05, 2021 at 07:32:00PM +0300, Vitaliy Gusev wrote: > Hi, > > > Does it mean madvise() doesn't work well in FreeBSD or test does something wrong? > > > > Your program does not exactly what you described above. There is a generic > > race to consume memory, and some specific details about madvise(2) on FreeBSD. > > > > >From the code, you do: > > - mmap anonymous private region > > - fork > > - both child and parent start touching the mmaped region. > > > > Two processes race to consume 1/2 of RAM on your system. If one of > > them happen to execute faster then another, you do get to the case where > > one of them does madvise(). But it could be that processes execute in > > lockstep, and try to eat all the memory before going to madvise(). > > Did you excluded this case? > I believe I did all things right. You can see sleeps that serialise execution. To check again I modified test and added time printing and use MADV_DONTNEED: > > Here is source http://cpp.sh/2rd4f > > I’ve run: > > $ ./mmapfork 2300 > mmap 0x801000000 pid 40628 > end 0x890c00000 len 0x8fc00000 > pid 40628 > pid 40629 > 40629: [1625500831] touch > 40629: [1625500832] sleep before madvise > 40629: [1625500833] madvise > 40629: [1625500834] Press enter to exit > 40628: [1625500845] touch > 40628: [1625500846] sleep before madvise > 40628: [1625500851] madvise > 40628: [1625500852] Press enter to exit > > And you can see that child started running in 11 seconds after parent had already called madvise() for all scope of touched memory. > > And finally in dmesg: > > pid 40629 (mmapfork), jid 0, uid 1001, was killed: out of swap space > > So the same result as I wrote in the first email. > > > Now, about the specific of madvise(MADV_FREE) on FreeBSD. Due to the way > > CoW is implemented with the shadow chain of objects, we cannot drop the > > top of the shadow chain, otherwise instead of returning zeroed pages next > > time, we would return content back in the time. It was relatively recent > > discovery, see bf5661f4a1af6931ec4b6, PR 240061. > > > Thanks, I will look at it. > > To explain it in simplified form, when there is potential old content > > under the CoW copy for the mapping, we cannot drop CoW-ed pages. This > > is the motivation why madvise(MADV_FREE) does nothing for your program. > > When you run two instances without fork, there is no previous content > > and no Cow, so madvise() can safely remove the pages from the object, > > and on the next access they are zero-filled. > > Do I understand right, that it should work with MADV_DONTNEED? But “dontneed" variant doesn’t work. DONTNEED does not allow system to free pages at all. It means that pages are less useful and can be paged out with higher priority. > > > > You can read more details in the referenced commit, as well as some musings > > about way to make it somewhat better. > > > > I must say, that trying to allocated 1/2 + 1/2 of RAM this way, on a system > > without swap, is the way to ask for troubles anyway. > I’ve just notify that other operation systems work well with that, whereas FreeBSD has troubles. Probably something in madvise() is not finished ? Well, yes, as I said, non-trivial shadow chains for MADV_FREE are not handled due to the 'old content revival' bug. For your specific case, the following patch might help (modulo bugs). But it is very specific for your example, for instance it would not work if you try to mark not the whole mapped area as _FREE, but only some significant part of it. We would need to start fragmenting map to handle such partial madvises better. commit 0392eb3c93b7dacc31dbdf8ec2fc40fa5ba67c62 Author: Konstantin Belousov Date: Mon Jul 5 21:53:22 2021 +0300 madvise(MADV_FREE): try harder to handle shadow chain In particular, collapse top object and see if there is no backing object after, which means that we would not revert to older content if drop the top object. diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index 1ac4ccf72f11..80abac223f29 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -3033,6 +3033,7 @@ vm_map_madvise( entry = vm_map_entry_succ(entry); for (; entry->start < end; entry = vm_map_entry_succ(entry)) { + vm_object_t obj; vm_offset_t useEnd, useStart; if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) != 0) @@ -3046,9 +3047,16 @@ vm_map_madvise( * backing object can change. */ if (behav == MADV_FREE && - entry->object.vm_object != NULL && - entry->object.vm_object->backing_object != NULL) - continue; + (obj = entry->object.vm_object) != NULL && + obj->backing_object != NULL) { + VM_OBJECT_WLOCK(obj); + if ((obj->flags & OBJ_DEAD) != 0) + continue; + vm_object_collapse(obj); + VM_OBJECT_WUNLOCK(obj); + if (obj->backing_object != NULL) + continue; + } pstart = OFF_TO_IDX(entry->offset); pend = pstart + atop(entry->end - entry->start); From nobody Mon Jul 5 19:31:26 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7AD007CB036 for ; Mon, 5 Jul 2021 19:31:29 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJbQn2pvrz3h3D; Mon, 5 Jul 2021 19:31:29 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lf1-x132.google.com with SMTP id t17so34257711lfq.0; Mon, 05 Jul 2021 12:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=/XUINDp8zFuPXvJ4NqjX12lB8J6cqXRAyM+nuDlB5t0=; b=HcbrTVxGWnb9/zG0LimeR+Oh+EPXAwEehFHgQrXu754ZRcuAyS5KmzBIJxKfNb0RBj skz6/PoUXTV2gsqn9qr16cpyVCSQ2blCRDerTGsvJyAFpu+s1m/Yudd2fiG1pV1X38bo pWq9gajqRRc6K8yYqigDKyijFwinZP9HGJdnjH/MM1u68Jj5S3SXkm3VMvcbzJJW+pq7 F4garGPfmZqv5iILT0Dyv588zYqWF5sumgMLh6/3u5kjviphEkk5gz++RFdRMHSY5dhA ecpyMByalbceQjtcmAwV6/oMr5b1oZ/H6y+HvX3KcBTtY8ZX+OQoMKr7Pz13b0qt+32Q hxkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=/XUINDp8zFuPXvJ4NqjX12lB8J6cqXRAyM+nuDlB5t0=; b=trM4X5+kY6Tg8Qxg0aMsKh1gIhtobXjV6uCEcGHTkaTnT+F3unlLrCFgDoOudcEqj5 kYoBv6af0pFCrfsyEnlq8al/V4yZXq7HVbojXxm8dMOKoFzcXMhIe+Nhnuckzcigz8qT SeK39ZcxBFyw+6luxfRUjjjnRCwPCcooHkd4YDF+ypyygrBpiQDAAGJnRd7jOMnqB0wH pl/+/2a5ll8sfUQ/D6gRQociN54f85IQFxmb1lhCnGf3C5/+SVBrDv4fjLSawALWylHe 2K/HOMmXix4uPmEZGGiRzXbJOaz2atATcQCtU3ZeMqym7nYFE8x+fXuXYErdqolX5UCI GRIg== X-Gm-Message-State: AOAM530j/rweTKrbxIRTk8toxS126gJ6NO8B3yLlYmNXnHqzAKGsyjvy KmmVppkyA8Eti4S/GKU4Aig= X-Google-Smtp-Source: ABdhPJy7hAuPwTsARI2x/7PQQd64+B4evLkner610ne2CjmMP5xW6SJeC2G9cwIL3reHxDWiEX9NWw== X-Received: by 2002:a05:6512:b26:: with SMTP id w38mr12182517lfu.227.1625513488071; Mon, 05 Jul 2021 12:31:28 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id a28sm1260825lfm.240.2021.07.05.12.31.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Jul 2021 12:31:27 -0700 (PDT) From: Vitaliy Gusev Message-Id: <59F26FB5-3A88-42C5-816F-B877977CBA6B@gmail.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_9276BB31-643D-47F6-84E2-082A874DDA5D" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Date: Mon, 5 Jul 2021 22:31:26 +0300 In-Reply-To: <57BCE463-6200-4F83-A321-2F0444E7F063@gmail.com> Cc: freebsd-hackers@freebsd.org, Mark Johnston To: Konstantin Belousov References: <0A95973D-254A-4574-8DC7-9F515F60B873@gmail.com> <57BCE463-6200-4F83-A321-2F0444E7F063@gmail.com> X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GJbQn2pvrz3h3D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_FROM(0.00)[]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_9276BB31-643D-47F6-84E2-082A874DDA5D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Further investigation shown that if use MADV_DONTNEED in simple mmap = test program and run it twice (second after the first finishes = madvise()), then memory is not freed at all and first instance is = killed. If change MADV_DONTNEED flag with MADV_FREE, then memory is freed. Code is: #include #include #include #include #include #include #include int main(int argc, char *argv[]) { size_t len =3D (size_t)(argc > 1 ? atoi(argv[1]) : 1024) * 1024 = * 1024; uint8_t *ptr, *end; uint8_t *p; int pagesz =3D (1<<12); ptr =3D (uint8_t *)mmap(NULL, len, PROT_WRITE | PROT_READ, = MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); if (ptr =3D=3D MAP_FAILED) err(1, "cannot mmap"); end =3D ptr + len; p =3D ptr; while (p < end) { *(uint64_t *)p =3D 1; p +=3D pagesz; } sleep(1); printf("madvise\n"); p =3D ptr; while (p < end) { int error; error =3D madvise(p, pagesz, MADV_DONTNEED); if (error) { err(1, "cannot madvise"); } p +=3D pagesz; } printf("press Enter to exit\n"); getchar(); } --Apple-Mail=_9276BB31-643D-47F6-84E2-082A874DDA5D-- From nobody Mon Jul 5 19:47:41 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 335097CDEBA for ; Mon, 5 Jul 2021 19:47:44 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJbnX0l3Yz3kRS; Mon, 5 Jul 2021 19:47:43 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lf1-x12a.google.com with SMTP id c28so10797045lfp.11; Mon, 05 Jul 2021 12:47:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=86JzPH9MLxZzDqf63gQaP0qBE2RRVkOK4wobC590+gU=; b=UExSQOaeVsT+Kc39OYLo1HVjqUf8VBZ2ukgjXH0MENMDHYyxPvrkzYiLzwv33A+HrL deaZhnLRDxDgi1XPSUBk1rVE3KCu0SxKltSGeGQ5fy+K+fJGoVCu8WpfRZK2JLqsqIIG 9aeakDUQGxiTLzXF74Dz5iEzsK1xgwPHE958nEEDRCktM/hDU8dalb0TV0KZeWv3spf2 F7E8f+/crhBXikIp44mHERJw+NNyeG/1zdZwwc0I8GCGBVR/wdVyuCm9lUsjrFUD+42+ +fvXwecNs7yJWW4CxMEdBIUYHswl952NnMsnSG9+cr49VwnTtEybDkxLVQ/KJy9s84im N8Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=86JzPH9MLxZzDqf63gQaP0qBE2RRVkOK4wobC590+gU=; b=EH3Hmsp5D0Xskk75JIuKkZNw1hHZIW5V8IrrMHhjkWU+plZoMt6FqXqsyO/bmG9Hdb vlkns5sqzHAUhZNTo0VmKfGw/F0gcKKKH6QgHyfAFj0ZK4Yyaevehy0F2yXLsZkqL6/g SKZXpd/v7blCoTIWZlpzgKnW9T5I5fSQ1+KcPhnM3H25X023MDTLZlKN4eU9EHLEskpw OIGG3GRjdDThPyewRaLZjaCheJ3+S5/XgdCABIcsH8XptmkdHs/KG/PPaZ58iG7X2+JM Fh5nsv13feCVh/lvjGch9xaTrfKrWXrYX7JXJvbO5W7XRAGEYfMYWJwIfdYXB7qyjNj3 +dPg== X-Gm-Message-State: AOAM531DzljC0ugehNEwb0SmjSpFvWm9hfoWIG79gB1vSNaXbtjZ9fW+ 9WqqsrZRw7JgLVDDL0248pM= X-Google-Smtp-Source: ABdhPJzXQ7IYg0aHlEpQPSpP8cPwKWDWTYfkN8W7O6jD4W3ABCpRmJj8/8tgvPaqzoMH8+X/GN60Zw== X-Received: by 2002:a19:6803:: with SMTP id d3mr11696968lfc.235.1625514461909; Mon, 05 Jul 2021 12:47:41 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id u5sm1173340lfg.268.2021.07.05.12.47.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Jul 2021 12:47:41 -0700 (PDT) From: Vitaliy Gusev Message-Id: <2A7A1E60-766D-427B-8288-E92B4090022E@gmail.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_82CC2F6D-5182-4E17-96FC-1768BCE1AE35" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Date: Mon, 5 Jul 2021 22:47:41 +0300 In-Reply-To: Cc: freebsd-hackers@freebsd.org, gljennjohn@gmail.com, Mark Johnston , Peter Grehan To: Konstantin Belousov References: <20210703065420.6dbafb5f@ernst.home> <2390FA9B-319E-45D4-BEA7-10878E43AD4B@gmail.com> X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GJbnX0l3Yz3kRS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_82CC2F6D-5182-4E17-96FC-1768BCE1AE35 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 5 Jul 2021, at 21:54, Konstantin Belousov = wrote: >=20 > On Mon, Jul 05, 2021 at 07:32:00PM +0300, Vitaliy Gusev wrote: >=20 > DONTNEED does not allow system to free pages at all. It means that = pages > are less useful and can be paged out with higher priority. >=20 That caused behaviour that memory is not freed!=20 Is there a reason why FreeBSD behaviour for MADV_DONTNEED is different = than in Linux and illumos ? Why it needs swap and cannot be easily = dropped ? As Mark wrote in another thread: Indeed, in FreeBSD madvise() never reclaims pages. I'm not sure which hint is being provided here, but MADV_DONTNEED and MADV_FREE both prioritize reclamation of the pages backing the specified virtual address range. The difference between the two is that MADV_FREE permits dirty pages to be freed without paging out their contents first and so is destructive. That is why I expected the same behaviour for both. >>>=20 >>> You can read more details in the referenced commit, as well as some = musings >>> about way to make it somewhat better. >>>=20 >>> I must say, that trying to allocated 1/2 + 1/2 of RAM this way, on a = system >>> without swap, is the way to ask for troubles anyway. >> I=E2=80=99ve just notify that other operation systems work well with = that, whereas FreeBSD has troubles. Probably something in madvise() is = not finished ? >=20 > Well, yes, as I said, non-trivial shadow chains for MADV_FREE are not > handled due to the 'old content revival' bug. For your specific case, = the > following patch might help (modulo bugs). Thanks, I will try and report result. =E2=80=94=E2=80=94 Vitaliy Gusev --Apple-Mail=_82CC2F6D-5182-4E17-96FC-1768BCE1AE35-- From nobody Mon Jul 5 22:21:17 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 171BE11DE055 for ; Mon, 5 Jul 2021 22:21:23 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJgBp719Kz4cmJ; Mon, 5 Jul 2021 22:21:22 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lf1-x12d.google.com with SMTP id p16so7326296lfc.5; Mon, 05 Jul 2021 15:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=IIlhWGwK0wegr8qrlm3GRTH3aGcLbD4xXKlBBegcnmc=; b=jcazQ6CNpd7b63pKkyMg2ysceBORFcfKAYWXS2GVcyOL1zo2XHE5BAqZ7b38oZr6s/ lVIVz+Vz2CpQ4ssXnrdcrT2q7LgkvTevvkUuAVZtSnwLwTBTKCZKUKNcV6BM7KizToJZ D6aZAzQFzYwFczOLK6OWwl6WM981Xvi6dhV50L9oqTwCwI7pNnbhVUEYf0G7A7CbZRvi wgJS0c04TvMbS3Sb8xw5LdZXiOH7UI8+v8ILoxtEAKOUXqcyWivS8WFwCHNSvTmS6Yqs NcBihV/WsPJIizx0GGo7i6/IjgR4/mCjXCevo1CO7qX8NVAvZkcKPYpEtF8c9ZdvRepC YMOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=IIlhWGwK0wegr8qrlm3GRTH3aGcLbD4xXKlBBegcnmc=; b=Tr1gsnGCN/tJhgEhPZZaHph/tFSZTdlwiDKzS9CRJkUm7AjrSgctV5YgnhStqLyW1e 1SksV59KDLskJdB1MoFDhULZi4oXihBsbikPjnnWuqQm+rEagn16TANTXY32nTmbeurW lNd8y2DRF16hXJSrXD1F8f0Evd01cUxE0sNCXTc4InNZujQuqvdIB01W03RrO9VLPcTU WzkxAFs2geY4t3M2kR94m32vJ2QPFpDkUbMn+JesJwRBvXsuCn9Mll1HssyvtyuJmhID zWG/5cnN8FOlKrwWl47qbxfPv4CCDgIRkGu+ELEC08qSqPVeKwaAIdaWUj/bJ6tFqVW8 xqiA== X-Gm-Message-State: AOAM530DimWJ3ys+QjUHYduXW94eDJ7JstjNGx/m28U9pW97lh37kNeG tXYC/meZBu+LCu/tGrlKl9M= X-Google-Smtp-Source: ABdhPJyXd1e8IvXHNzKmHb+4I/NF4bxoDaHyuc8KMtxeRPw03FgtT3l9Af/cm89Ddcl1Y711ST0vMg== X-Received: by 2002:ac2:55a7:: with SMTP id y7mr12225974lfg.179.1625523681153; Mon, 05 Jul 2021 15:21:21 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id p15sm370977lfh.161.2021.07.05.15.21.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Jul 2021 15:21:20 -0700 (PDT) From: Vitaliy Gusev Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_840FE2D4-B08C-45F8-8446-FEF886AF0FC5" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Date: Tue, 6 Jul 2021 01:21:17 +0300 In-Reply-To: <2A7A1E60-766D-427B-8288-E92B4090022E@gmail.com> Cc: freebsd-hackers@freebsd.org, gljennjohn@gmail.com, Mark Johnston , Peter Grehan To: Konstantin Belousov References: <20210703065420.6dbafb5f@ernst.home> <2390FA9B-319E-45D4-BEA7-10878E43AD4B@gmail.com> <2A7A1E60-766D-427B-8288-E92B4090022E@gmail.com> X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GJgBp719Kz4cmJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_FROM(0.00)[]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_840FE2D4-B08C-45F8-8446-FEF886AF0FC5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Comments are below, > commit 0392eb3c93b7dacc31dbdf8ec2fc40fa5ba67c62 > Author: Konstantin Belousov > > Date: Mon Jul 5 21:53:22 2021 +0300 >=20 > madvise(MADV_FREE): try harder to handle shadow chain >=20 > In particular, collapse top object and see if there is no backing = object > after, which means that we would not revert to older content if = drop the > top object. >=20 > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > index 1ac4ccf72f11..80abac223f29 100644 > --- a/sys/vm/vm_map.c > +++ b/sys/vm/vm_map.c > @@ -3033,6 +3033,7 @@ vm_map_madvise( > entry =3D vm_map_entry_succ(entry); > for (; entry->start < end; > entry =3D vm_map_entry_succ(entry)) { > + vm_object_t obj; > vm_offset_t useEnd, useStart; >=20 > if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) !=3D = 0) > @@ -3046,9 +3047,16 @@ vm_map_madvise( > * backing object can change. > */ > if (behav =3D=3D MADV_FREE && > - entry->object.vm_object !=3D NULL && > - entry->object.vm_object->backing_object !=3D = NULL) > - continue; > + (obj =3D entry->object.vm_object) !=3D NULL = && > + obj->backing_object !=3D NULL) { > + VM_OBJECT_WLOCK(obj); > + if ((obj->flags & OBJ_DEAD) !=3D 0) > + continue; Here is object is left locked, however I didn=E2=80=99t met this = condition. > + vm_object_collapse(obj); > + VM_OBJECT_WUNLOCK(obj); > + if (obj->backing_object !=3D NULL) > + continue; After testing it looks that backing_object is not null here, memory is = not freed and mmapfork test program is still killed. Any ideas? =E2=80=94=E2=80=94 Vitaliy Gusev= --Apple-Mail=_840FE2D4-B08C-45F8-8446-FEF886AF0FC5-- From nobody Wed Jul 7 17:47:47 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4029D11F48EE for ; Wed, 7 Jul 2021 17:48:04 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "m5p.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKn2W2ZNWz4XR5 for ; Wed, 7 Jul 2021 17:48:00 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::26] (court.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:26]) (authenticated bits=0) by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPSA id 167HllNo035329 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 7 Jul 2021 13:47:52 -0400 (EDT) (envelope-from george+freebsd@m5p.com) To: FreeBSD Hackers From: George Mitchell Subject: Periodic rant about SCHED_ULE Message-ID: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> Date: Wed, 7 Jul 2021 13:47:47 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="HF7V0AghedMoWNFDpuKNTarcdpkVGOeR5" X-Spam-Status: No, score=0.2 required=10.0 tests=HELO_MISC_IP,HELO_NO_DOMAIN autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com X-Rspamd-Queue-Id: 4GKn2W2ZNWz4XR5 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of george@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=george@m5p.com X-Spamd-Result: default: False [-1.67 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; HAS_ATTACHMENT(0.00)[]; TO_DN_ALL(0.00)[]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[74.104.188.4:from]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[freebsd]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.94)[0.939]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[m5p.com]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[74.104.188.4:from:127.0.2.255]; NEURAL_SPAM_LONG(0.79)[0.789]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --HF7V0AghedMoWNFDpuKNTarcdpkVGOeR5 Content-Type: multipart/mixed; boundary="JEQlsITH6BsjloiA4uygFg539pzKZdt9A"; protected-headers="v1" From: George Mitchell To: FreeBSD Hackers Message-ID: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> Subject: Periodic rant about SCHED_ULE --JEQlsITH6BsjloiA4uygFg539pzKZdt9A Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable CPU: AMD Ryzen 5 2600X Six-Core Processor (3600.10-MHz K8-class CPU) (12 threads). FreeBSD 12.2-RELEASE-p7 r369865 GENERIC amd64 (SCHED_ULE) vs FreeBSD 12.2-RELEASE-p7 r369865 M5P amd64 (SCHED_4BSD). Comparing "make buildworld" time with misc/dnetc running vs not running. (misc/dnetc is your basic 100% compute-bound task, running at nice 20.) Three out of the four combinations build in roughly four hours, but SCHED_ULE with dnetc running takes close to twelve! (And that was overnight with basically nothing else running.) This is an even worse disparity than I have seen in previous releases. Based on my total lack of an exact understanding of how loadable kernel modules work, I nevertheless hypothesize that it should be possible for the scheduler to be a loadable kernel module, given that the interface to the schedule seems to have been carefully thought out and scrupulously observed by both ULE and 4BSD. Perhaps the system could start with a dummy scheduler that is just barely functional enough to get the appropriate module loaded and have it supersede the dummy scheduler? Then switching between the schedulers would become much less of a hassle than it is currently. Why, if it is configurable, it could even be compatible with freebsd-update! I've been ranting about this for years now, and I've had my say -- but no one has ever answered my question about what workload SCHED_ULE is best for, though numerous people have claimed that it's better than SCHED_4BSD for -- some rumored workload or other. -- George --JEQlsITH6BsjloiA4uygFg539pzKZdt9A-- --HF7V0AghedMoWNFDpuKNTarcdpkVGOeR5 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAmDl6MMFAwAAAAAACgkQwRES3m+p4fkP MRAAutre1e/bqhY2w1fQdiktJc6bo7+kFgW1V8qxTNumUXfyZCgL6DCALhKrDkBaAZsuvwWFSAaK ASjGOTdlPA/fZ0tr2HWH77Z2kbj1RP7ce9zDP5PDVWZaPeD7arnAJSBvD0K9kiI/sAVBRW80WH4C zj1rW4/XqcJN2/YPsjLfVTbToqEPJN6XpodInYrpKBF2W47hTcHF6qxUkaa/7J5gIDOY2ad7c35q tuCQKlsoFPpD6l/UYXevO5pe/MVEX5hJLHrPhCUbqZEgeVMltqn+V76DVcywO/x0cBj65k3TETus tAcZebbKuvII9WBfM58Kt+zMRc3ezU+7pN6FOwJCj89vJ6OLO19LAXQ2zvLmSDYRz+OjxK4Z3IZk 3MWpc2qwOoNsABp9bg8pvbihNoug9bM/qhbyFP1+nbyHvbpgOIid9K5jub20TNPEN/j0cSGUQaed rs5rZlv7XuOMspr1hq5fra26M/nh3li31mltfNA2qHoBYecpiBY3x+CfDGydOIVbl8+ed/A+tQAX /3mVORXsLry7jSB+dBSakJ4caR97Ehc/nWSFFGn+WlYtTURi7mGFI/axm/RGKdR/TSasS6MlPJLg lZbSOTvE/i/5N2+162qgLEuYuqAjPUnlIvJ8yzN9nHFMdSTSr3uuyeqaXaketRmP9h7k7wxCdzQg aJ8= =0sGY -----END PGP SIGNATURE----- --HF7V0AghedMoWNFDpuKNTarcdpkVGOeR5-- From nobody Wed Jul 7 18:18:35 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D03F512228BE for ; Wed, 7 Jul 2021 18:18:41 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKnjs54mZz4drl for ; Wed, 7 Jul 2021 18:18:41 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ej1-x630.google.com with SMTP id c17so4707287ejk.13 for ; Wed, 07 Jul 2021 11:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=ahMVuPS//b9Di9wkPHsQ8UphefgFSTLxhSgP/SdEpNQ=; b=QEV74BZW9adqDhfShG1mfZcvbW+i/y8hdoKBEbWVSeH61FigvWitfTzNKKBiJf7k2M K5T4PvZYi1v+elKESDS4J1y+ARWKXGcCWQVsgx2PNBOoFL3mWnGnPKSb+5WI5RrzB+ra pzM/75CRKyOw4PGMt4I5PTvsJn4tO1czAOYmtTnM3bp55/v8FzjuvvlTpJ1vBeRZz0Si VRKo6Rntt1KqPSmw+oXkbU0AuPYmlmQYMYdoBlW6wZqvrYSrWwknS04Fn3kI/YVv1nBb 4RuvXjwhwTpGYcR8bJu/ki/fD/ERi0aQieXv8LprXQDqH9Hy3PAhxdF2s40jF7T4bvFz APvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=ahMVuPS//b9Di9wkPHsQ8UphefgFSTLxhSgP/SdEpNQ=; b=KBRJHv8QDZJL8fUpDoGSJ6USXRXj756YYQHArqPjWlwy+VqYVxjzgt3eDcIVOZriVH VDWnHkNa6OiHPhXIy/Tc5JnWuNaSqhaa6QXeQyjokClW2v1/7nkdLHZAULMl0xYgTMxv 04JOCDsc86EDguBuH+bl5vdFI+E5gudvzRYPaUlZRvZP0hXYJsO9+VqN5qFiGRF4XMwO MrKlsSRHZknNLcerMUQoMjQ1eJ9hWE8zgYKCzrcMh/YhvlmQ/32qG4YRLLXGAS9LcFQ7 6i7ymSJHzqYjdFjJzZnuQUX3Run2yo3js+XZ249Kf/iCPxVR7gMn0sPaBbzn+zjcVvhA ngjg== X-Gm-Message-State: AOAM5306P5oV+BASAwjku2yc+/Dh/ml0wf7KLmqaaWd9ghF1+vucT1Xe MLL01NOpHyNtN51sC1tj5H0= X-Google-Smtp-Source: ABdhPJx1w3mREflKGoCcr0ejuOVDNt+EildpPZQYQ73jRObpeSpb08BTCMmeBwfeDLmJyvJ2bDjSTw== X-Received: by 2002:a17:907:1c14:: with SMTP id nc20mr17424341ejc.140.1625681919567; Wed, 07 Jul 2021 11:18:39 -0700 (PDT) Received: from ernst.home (pd9e2360f.dip0.t-ipconnect.de. [217.226.54.15]) by smtp.gmail.com with ESMTPSA id k25sm7731409eds.77.2021.07.07.11.18.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 11:18:39 -0700 (PDT) Date: Wed, 7 Jul 2021 20:18:35 +0200 From: Gary Jennejohn To: George Mitchell Cc: FreeBSD Hackers Subject: Re: Periodic rant about SCHED_ULE Message-ID: <20210707181835.75601d54@ernst.home> In-Reply-To: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> References: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GKnjs54mZz4drl X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, 7 Jul 2021 13:47:47 -0400 George Mitchell wrote: > CPU: AMD Ryzen 5 2600X Six-Core Processor (3600.10-MHz K8-class CPU) > (12 threads). > > FreeBSD 12.2-RELEASE-p7 r369865 GENERIC amd64 (SCHED_ULE) vs > FreeBSD 12.2-RELEASE-p7 r369865 M5P amd64 (SCHED_4BSD). > > Comparing "make buildworld" time with misc/dnetc running vs not running. > (misc/dnetc is your basic 100% compute-bound task, running at nice 20.) > > Three out of the four combinations build in roughly four hours, but > SCHED_ULE with dnetc running takes close to twelve! (And that was > overnight with basically nothing else running.) This is an even > worse disparity than I have seen in previous releases. > > Based on my total lack of an exact understanding of how loadable > kernel modules work, I nevertheless hypothesize that it should be > possible for the scheduler to be a loadable kernel module, given that > the interface to the schedule seems to have been carefully thought out > and scrupulously observed by both ULE and 4BSD. Perhaps the system > could start with a dummy scheduler that is just barely functional > enough to get the appropriate module loaded and have it supersede the > dummy scheduler? Then switching between the schedulers would become > much less of a hassle than it is currently. Why, if it is configurable, > it could even be compatible with freebsd-update! > > I've been ranting about this for years now, and I've had my say -- but > no one has ever answered my question about what workload SCHED_ULE is > best for, though numerous people have claimed that it's better than > SCHED_4BSD for -- some rumored workload or other. -- George > IIRC there was talk about making the scheduler loadable in the early days. But that was years ago and I may be misrembering. I have a Ryzen 5 1600 with 6 cores, so older tech and "only" 3200MHz. I can do a clean buildworld on FreeBSD-14 using only 10 of the 12 SMTs in about 40 minutes using SCHED_4BSD. While still browsing the interwebs or watching a film etc. with no noticeable lags in performance. So, for my normal desktop usage SCHED_4BSD is the only way to go. -- Gary Jennejohn From nobody Wed Jul 7 20:56:55 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 373DC11F0F06 for ; Wed, 7 Jul 2021 20:57:07 +0000 (UTC) (envelope-from SRS0=EUh4=L7=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKsDf6tV2z3JmD for ; Wed, 7 Jul 2021 20:57:06 +0000 (UTC) (envelope-from SRS0=EUh4=L7=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B552D28411; Wed, 7 Jul 2021 22:56:57 +0200 (CEST) Received: from illbsd.quip.test (ip-94-113-69-69.net.upcbroadband.cz [94.113.69.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id C5BE928417; Wed, 7 Jul 2021 22:56:56 +0200 (CEST) Subject: Re: Periodic rant about SCHED_ULE To: gljennjohn@gmail.com, George Mitchell Cc: FreeBSD Hackers References: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> <20210707181835.75601d54@ernst.home> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <054b4735-7740-617d-6c61-c5b48ef1d85a@quip.cz> Date: Wed, 7 Jul 2021 22:56:55 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <20210707181835.75601d54@ernst.home> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GKsDf6tV2z3JmD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 07/07/2021 20:18, Gary Jennejohn wrote: > On Wed, 7 Jul 2021 13:47:47 -0400 > George Mitchell wrote: [..] >> I've been ranting about this for years now, and I've had my say -- but >> no one has ever answered my question about what workload SCHED_ULE is >> best for, though numerous people have claimed that it's better than >> SCHED_4BSD for -- some rumored workload or other. -- George >> > > IIRC there was talk about making the scheduler loadable in the early > days. But that was years ago and I may be misrembering. > > I have a Ryzen 5 1600 with 6 cores, so older tech and "only" 3200MHz. > > I can do a clean buildworld on FreeBSD-14 using only 10 of the 12 SMTs > in about 40 minutes using SCHED_4BSD. While still browsing the > interwebs or watching a film etc. with no noticeable lags in > performance. > > So, for my normal desktop usage SCHED_4BSD is the only way to go. I had some performance problems with VirtualBox as hypervisor on somewhat older Intel Xeon with 4 cores 8 threads. So I tested 4BSD and ULE - SCHED_4BSD had slightly better results than SCHED_ULE. I am also curious why ULE is the default. Where are some real world performance results for comparing the two FreeBSD schedulers. Kind regards Miroslav Lachman From nobody Wed Jul 7 21:01:54 2021 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 01B5311F2034 for ; Wed, 7 Jul 2021 21:02:07 +0000 (UTC) (envelope-from SRS0=CZ4p=L7=FreeBSD.org=otis@ns2.wilbury.net) Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKsLQ1BXsz3L3c for ; Wed, 7 Jul 2021 21:02:05 +0000 (UTC) (envelope-from SRS0=CZ4p=L7=FreeBSD.org=otis@ns2.wilbury.net) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 393BD45D18B for ; Wed, 7 Jul 2021 23:01:55 +0200 (CEST) From: Juraj Lutter Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Subject: Building -CURRENT on macOS big sur Message-Id: Date: Wed, 7 Jul 2021 23:01:54 +0200 To: hackers@freebsd.org X-Mailer: Apple Mail (2.3654.100.0.2.22) X-Spam-Status: No, score=-0.8 required=5.0 tests=BAYES_00,KHOP_HELO_FCRDNS, SPF_HELO_NONE,SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on ns2.wilbury.net X-Rspamd-Queue-Id: 4GKsLQ1BXsz3L3c X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=CZ4p=L7=FreeBSD.org=otis@ns2.wilbury.net has no SPF policy when checking 2a01:b200:0:1:f816:3eff:fecd:13e6) smtp.mailfrom=SRS0=CZ4p=L7=FreeBSD.org=otis@ns2.wilbury.net X-Spamd-Result: default: False [-1.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[otis@FreeBSD.org,SRS0=CZ4p=L7=FreeBSD.org=otis@ns2.wilbury.net]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:b200:0:1:f816:3eff:fecd:13e6:from]; ASN(0.00)[asn:44185, ipnet:2a01:b200::/32, country:SK]; R_DKIM_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.80)[-0.800]; FROM_NEQ_ENVFROM(0.00)[otis@FreeBSD.org,SRS0=CZ4p=L7=FreeBSD.org=otis@ns2.wilbury.net]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a01:b200:0:1:f816:3eff:fecd:13e6:from:127.0.2.255]; DMARC_NA(0.00)[FreeBSD.org]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] X-ThisMailContainsUnwantedMimeParts: N Hi, I want to build 14-CURRENT on macOS Big Sur host, using = tools/build/make.py, without any luck (ksh test is keeping segfaulting). Is there any good reading on this topic? Except from the traditional = google resources. Thanks otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Wed Jul 7 21:06:33 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0C9AD11F31CE for ; Wed, 7 Jul 2021 21:06:47 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKsRp2qFvz3MSs; Wed, 7 Jul 2021 21:06:46 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-pf1-x434.google.com with SMTP id d12so3428829pfj.2; Wed, 07 Jul 2021 14:06:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=22AeHMOLq7Osz219iOQfvNor5E+6BdMbVfjNpj3ioBc=; b=cJWQDtC4fvzzm13dnRGPlhNWJLwN63YPp6y086c23rxFnm75XHv8d9KRg2bbEYIoTz 0npnC1qFFhDBv+woymsLHeE1UwaFR0YfkpovhURFLZj+KxYwLO7ZAwS67csA7SETqWA/ QH2jmkw0RPVxFU2f7u29Wi2A3M0a7yyvwYtn7SfK74eV7XuuP7sPEZEyRK05IfZ2sOfd dymSq5v3nXgbYFMD/nju/KwFpcfWOk7DaeGB3LjH2jl+gCZjVVwySIJIJ6Ek1B+lE6k8 nSo0yu47K67EXEySXB0YGve18X5UFmatz8m/5HtAt21YGvh19lv+P0T/Ff7ZaDvkkcWF uwYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=22AeHMOLq7Osz219iOQfvNor5E+6BdMbVfjNpj3ioBc=; b=E1I8/3ypws7PzhJqoifYNMWF7FBtAyeZR4OEj0AusG18Scnzcci9dLjX/law2bVEr5 grUK4CFlvYb4l6+GzruNxW+/tIfdQUZqd3sCFcUqx1C8TVb/LvPVqKCu/17INSlwMvuC 1/bKxdW15QYtF7D1stzaHO6toBzX9y0LbfMBKXQ9GK9oXVaZxAsSe2D4AE15xWkl1zLQ p0rXSrQQG65G4O0/FOXA3T4xfvFyZpVG3At++3GwvYgLrotp+APR98ul+y04OjSqLpb0 RWUJuppRHs+sdDQSxBT3AhsBAspWm3SnJ01DaK3LrBvTd82683Cbr0gE0SGprMlJm8g6 /xwg== X-Gm-Message-State: AOAM533c0C9CeOAUPIGsOfvyrvjPi3/PK50loIIP+xPrDnS2sPTxyNuU UegXL1Ow8j96mGkL0TBrtrECZaBwukwKENsHiUSpaUwZZ0I= X-Google-Smtp-Source: ABdhPJx/hc/6tqqvOP3RQeZlNA9hwJb18BbxTcQHh4vLl3ZhjHaFekBUht4MS97mfu+MiCsTujGFewXPy9gxFNYlJjY= X-Received: by 2002:a65:640d:: with SMTP id a13mr23330500pgv.421.1625692003950; Wed, 07 Jul 2021 14:06:43 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Ryan Stone Date: Wed, 7 Jul 2021 17:06:33 -0400 Message-ID: Subject: clang build failure with WITH_LLVM_TARGET_BPF=yes To: FreeBSD Hackers Cc: Dimitry Andric Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GKsRp2qFvz3MSs X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=cJWQDtC4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::434 as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [-1.43 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::434:from]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-0.43)[-0.427]; NEURAL_SPAM_SHORT(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::434:from:127.0.2.255]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::434:from]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-hackers]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N I have the following option enabled in src.conf, and I'm getting this build failure in clang. Disabling the option seems to fix the build. ld: error: undefined symbol: llvm::initializeBPFAdjustOptPass(llvm::PassRegistry&) >>> referenced by BPFTargetMachine.cpp >>> BPFTargetMachine.o:(LLVMInitializeBPFTarget) in archive /usr/obj/srcpool/src/rstone/freebsd/amd64.amd64/tmp/obj-tools/lib/clan g/libllvm/libllvm.a ld: error: undefined symbol: llvm::initializeBPFCheckAndAdjustIRPass(llvm::PassRegistry&) >>> referenced by BPFTargetMachine.cpp >>> BPFTargetMachine.o:(LLVMInitializeBPFTarget) in archive /usr/obj/srcpool/src/rstone/freebsd/amd64.amd64/tmp/obj-tools/lib/clan g/libllvm/libllvm.a ld: error: undefined symbol: llvm::createBPFCheckAndAdjustIR() >>> referenced by BPFTargetMachine.cpp >>> BPFTargetMachine.o:((anonymous namespace)::BPFPassConfig::addIRPasses()) in archive /usr/obj/srcpool/src/rstone/freebsd/amd64. amd64/tmp/obj-tools/lib/clang/libllvm/libllvm.a ld: error: undefined symbol: llvm::createBPFAdjustOpt() >>> referenced by BPFTargetMachine.cpp >>> BPFTargetMachine.o:(std::__1::__function::__func, void (llvm::PassManagerBuilder const&, llvm::legac y::PassManagerBase&)>::operator()(llvm::PassManagerBuilder const&, llvm::legacy::PassManagerBase&)) in archive /usr/obj/srcpool/src/rstone/freeb sd/amd64.amd64/tmp/obj-tools/lib/clang/libllvm/libllvm.a ld: error: undefined symbol: llvm::BPFAdjustOptPass::run(llvm::Module&, llvm::AnalysisManager&) >>> referenced by BPFTargetMachine.cpp >>> BPFTargetMachine.o:(llvm::detail::PassModel >::run(llvm::Module&, llvm::AnalysisManager&)) in archive /usr/obj/srcpool/src/rstone/freebsd/amd64.amd64/tmp/obj -tools/lib/clang/libllvm/libllvm.a From nobody Wed Jul 7 21:49:19 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6D63A122238A for ; Wed, 7 Jul 2021 21:49:28 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKtP41SChz3jdN for ; Wed, 7 Jul 2021 21:49:27 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.16.1/8.16.1) with ESMTPS id 167LnKqm022389 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 7 Jul 2021 14:49:20 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.16.1/8.16.1/Submit) id 167LnJms022388; Wed, 7 Jul 2021 14:49:19 -0700 (PDT) (envelope-from sgk) Date: Wed, 7 Jul 2021 14:49:19 -0700 From: Steve Kargl To: Miroslav Lachman <000.fbsd@quip.cz> Cc: gljennjohn@gmail.com, George Mitchell , FreeBSD Hackers Subject: Re: Periodic rant about SCHED_ULE Message-ID: <20210707214919.GA22332@troutmask.apl.washington.edu> References: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> <20210707181835.75601d54@ernst.home> <054b4735-7740-617d-6c61-c5b48ef1d85a@quip.cz> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <054b4735-7740-617d-6c61-c5b48ef1d85a@quip.cz> X-Rspamd-Queue-Id: 4GKtP41SChz3jdN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, Jul 07, 2021 at 10:56:55PM +0200, Miroslav Lachman wrote: > On 07/07/2021 20:18, Gary Jennejohn wrote: > > On Wed, 7 Jul 2021 13:47:47 -0400 > > George Mitchell wrote: > > [..] > > > > I've been ranting about this for years now, and I've had my say -- but > > > no one has ever answered my question about what workload SCHED_ULE is > > > best for, though numerous people have claimed that it's better than > > > SCHED_4BSD for -- some rumored workload or other. -- George > > > > > > > IIRC there was talk about making the scheduler loadable in the early > > days. But that was years ago and I may be misrembering. > > > > I have a Ryzen 5 1600 with 6 cores, so older tech and "only" 3200MHz. > > > > I can do a clean buildworld on FreeBSD-14 using only 10 of the 12 SMTs > > in about 40 minutes using SCHED_4BSD. While still browsing the > > interwebs or watching a film etc. with no noticeable lags in > > performance. > > > > So, for my normal desktop usage SCHED_4BSD is the only way to go. > > I had some performance problems with VirtualBox as hypervisor on somewhat > older Intel Xeon with 4 cores 8 threads. So I tested 4BSD and ULE - > SCHED_4BSD had slightly better results than SCHED_ULE. > I am also curious why ULE is the default. Where are some real world > performance results for comparing the two FreeBSD schedulers. > I made those measurements more than a decade ago, and reported my findings in either freebsd-hackers or freebsd-current mailing list. Write a classic boss-worker MPI numerical simulation, where the workers are compute bound. Start the MPI simulation requesting NCPU+1 images with NCPU being the number of available cpus. Each worker will be assigned to a cpu. Then this leads to a worker and the boss image sharing a cpu. Due to cpu affinity, these two then ping-pong on that cpu. I haven't repeated these measurement in a long time. -- Steve From nobody Thu Jul 8 06:41:23 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D4958122139C for ; Thu, 8 Jul 2021 06:41:26 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GL6Bt5Km8z3KtX for ; Thu, 8 Jul 2021 06:41:26 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ed1-x534.google.com with SMTP id v1so6927811edt.6 for ; Wed, 07 Jul 2021 23:41:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=YW/QfjWvO5QHUT6m9eU10+qr95BzYU7LvZaRyNJ9zKo=; b=kt49RXJW84vhJoUq6LKv4WodOuBNzyoVnXbpyF9L49RtnCiGg94XNNsUP2/cARSoxq XsjtzuZSpi4a7Ehx5kLSNHAE0xwbJSbRUboXR5T9Dsa63/gYC5Z96WneLS5e+A1HDofa 1VHR0aN9M03wuJQPgvhaHl4XDQ2A4BGYHDLUlIuE8SVLEMGbhiegsTaq9bPV0FLUrY6t idSYmZ+D3dYLo+A3rCS2NM1R6MtZ9RJE459MHb/IByrBF6CJZeGLjuO4YWfqfWQ28d7H FallMBtXVuopHlrh7mqUjllrlBhl2i9xxrGVPoYKmhu6nuzOqWn+nfxEXA9ZqdNFnFen h1qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=YW/QfjWvO5QHUT6m9eU10+qr95BzYU7LvZaRyNJ9zKo=; b=Zyhs627cv31COYPgkRadiwjxEt9/NYXoKwJnI9lKV4/p1RgDdRpxuNQL14uoFxLAJG TSrioQWRGD9gIwzbFcV1sHObLWEb5GWOZvtfnfg+1Rqg5G7ENRuL/H/CuFGOdyR6lj5M HgIsbYh6aSgCLohSh4ARog+n36Pavdv5CISqv6M7y82KMj5S4n9xYaaPTIbevXKgOMuZ xEwiX1Rd2PkYXfM2ZBQJjdXaZwNBYvPUH0rzIdimhubZYwuGwcXp1x/f5g7cdEz9M+rT LQKNM6DuezAFAXNRgTIDKdM7HGThsoruHTeVEoWIkQLUqMo4W+mBlkact39uU5AS7eYJ rlvQ== X-Gm-Message-State: AOAM531Cv8Sgoa4eGSyWq9VV0O2qddFgAA5VoGfvyg5rQSLQR+WxuWDy AIVqg3nc3XUMfYNXwhh1a5e4tORFY2xV4w== X-Google-Smtp-Source: ABdhPJza8SXpU3CzAqb3PqcORVzgS39cRj+gTqdw0IszSlhDz3EjHNI1lUzf3TpFDM/ehBmsGfmaiQ== X-Received: by 2002:aa7:c74e:: with SMTP id c14mr7901143eds.40.1625726485348; Wed, 07 Jul 2021 23:41:25 -0700 (PDT) Received: from ernst.home (pd9e2360f.dip0.t-ipconnect.de. [217.226.54.15]) by smtp.gmail.com with ESMTPSA id gg4sm416274ejb.87.2021.07.07.23.41.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 23:41:24 -0700 (PDT) Date: Thu, 8 Jul 2021 08:41:23 +0200 From: Gary Jennejohn To: Steve Kargl Cc: Miroslav Lachman <000.fbsd@quip.cz>, George Mitchell , FreeBSD Hackers Subject: Re: Periodic rant about SCHED_ULE Message-ID: <20210708064123.22ae2daf@ernst.home> In-Reply-To: <20210707214919.GA22332@troutmask.apl.washington.edu> References: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> <20210707181835.75601d54@ernst.home> <054b4735-7740-617d-6c61-c5b48ef1d85a@quip.cz> <20210707214919.GA22332@troutmask.apl.washington.edu> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GL6Bt5Km8z3KtX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, 7 Jul 2021 14:49:19 -0700 Steve Kargl wrote: > On Wed, Jul 07, 2021 at 10:56:55PM +0200, Miroslav Lachman wrote: > > On 07/07/2021 20:18, Gary Jennejohn wrote: > > > On Wed, 7 Jul 2021 13:47:47 -0400 > > > George Mitchell wrote: > > > > [..] > > > > > > I've been ranting about this for years now, and I've had my say -- but > > > > no one has ever answered my question about what workload SCHED_ULE is > > > > best for, though numerous people have claimed that it's better than > > > > SCHED_4BSD for -- some rumored workload or other. -- George > > > > > > > > > > IIRC there was talk about making the scheduler loadable in the early > > > days. But that was years ago and I may be misrembering. > > > > > > I have a Ryzen 5 1600 with 6 cores, so older tech and "only" 3200MHz. > > > > > > I can do a clean buildworld on FreeBSD-14 using only 10 of the 12 SMTs > > > in about 40 minutes using SCHED_4BSD. While still browsing the > > > interwebs or watching a film etc. with no noticeable lags in > > > performance. > > > > > > So, for my normal desktop usage SCHED_4BSD is the only way to go. > > > > I had some performance problems with VirtualBox as hypervisor on somewhat > > older Intel Xeon with 4 cores 8 threads. So I tested 4BSD and ULE - > > SCHED_4BSD had slightly better results than SCHED_ULE. > > I am also curious why ULE is the default. Where are some real world > > performance results for comparing the two FreeBSD schedulers. > > > > I made those measurements more than a decade ago, and > reported my findings in either freebsd-hackers or > freebsd-current mailing list. > > Write a classic boss-worker MPI numerical simulation, where the > workers are compute bound. Start the MPI simulation requesting > NCPU+1 images with NCPU being the number of available cpus. > Each worker will be assigned to a cpu. Then this leads to a > worker and the boss image sharing a cpu. Due to cpu affinity, > these two then ping-pong on that cpu. > > I haven't repeated these measurement in a long time. > I feel that I have to add that the 40 minute buildworld isn't due to SCHED_4BSD alone, but that I have many features which I don't need disabled in src.conf. I'm pretty sure that a buildworld with every feature enabled would take a lot longer on my system. -- Gary Jennejohn From nobody Thu Jul 8 07:19:07 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3B88E1225C1A for ; Thu, 8 Jul 2021 07:19:15 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GL72W0xvKz3PX4 for ; Thu, 8 Jul 2021 07:19:14 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-lf1-x12d.google.com with SMTP id 8so556538lfp.9 for ; Thu, 08 Jul 2021 00:19:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=svxLhvZ+EgOOnXXFK3w6ZrsrriMdPHYutUgTAKb8bS4=; b=AE9MlaAn/JoS+ric/fQ9cFX9YjPdRlZJ1kd9uOyuk/hjB8ymS+zppqEAaXyM8rahxh K83KCtIUHEc488kqzF0ZoimaDq7U94Y7vW3hLiXq18iDzwNv1LQa3h8wWEKIiU2wkDqL VFLkbJzEjQulvVY+eItMXNtze61/UWWf60+hNNJtvsnykHqTUcbxyNXljuPXXhN/gGVq YvydJ9cawbav0PFBVt2siVdGqYRyHSgAppwMMLZY2XVVIbS4KDlh56ODLmzphS6iMq9f k/MCti9A3Bqnb8knyQYRCE0AnJ0b3gWuy4WGUh1LcBeEb+neJeEj9u8/mph3/eLHduFj F2yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=svxLhvZ+EgOOnXXFK3w6ZrsrriMdPHYutUgTAKb8bS4=; b=BgB0d/qVn56o3luu2oo2xWEyA8P+W2lKxXv0KnnaOp0KYQtQxnGSF4mwcv4Lm9CayN WTweCFF0FP+4PsEzyWXwiM4m8UwkfxEOPl+NPxJTkIaEhchRNM+e9oapuZdVs6UylBpU XkTAz0aiCTQHDgUL65Mn2jrCDPFRbOmlGdh610VB1xzdds16w7STRUarH5xhCjt0eZpa 45rZDhAOZPmi1n1VLMjeJJhP8WCT4LYqbbpZ7xA0KThBgUeZcnYXUD7mKk16BPPE3NTl K4EWfkvMdbiGj9ePFxik1bV55en6K0yz3q56V5sXl7GN90+9JmUwjn3hER4dbJV5Z9Kl w/aw== X-Gm-Message-State: AOAM5321Uen0blS/qSmZdQdoNy29gMzVLqi4VmoWFNw2FVNzNh15qsBw /stZVZASSJZGfTr5TrW/Fu0DXwH+mr4= X-Google-Smtp-Source: ABdhPJzlJRSkRL+i6jzwG+H7+0+UU73bn0yGtclqrl1ccHpy1PKSLiZ8StAmz8S9OzvFXrhWdxCKZg== X-Received: by 2002:a05:651c:1ac:: with SMTP id c12mr8487994ljn.179.1625728753725; Thu, 08 Jul 2021 00:19:13 -0700 (PDT) Received: from rimwks.local ([2001:470:1f15:3d8:7285:c2ff:fe37:5722]) by smtp.gmail.com with ESMTPSA id y6sm153646lje.21.2021.07.08.00.19.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 00:19:13 -0700 (PDT) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Thu, 8 Jul 2021 10:19:07 +0300 To: George Mitchell Cc: FreeBSD Hackers Subject: Re: Periodic rant about SCHED_ULE Message-ID: <20210708101907.0be3a3c2@rimwks.local> In-Reply-To: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> References: <13445948-7804-20b4-4ae6-aaac14d11e87@m5p.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd13.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GL72W0xvKz3PX4 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, 7 Jul 2021 13:47:47 -0400 George Mitchell wrote: > CPU: AMD Ryzen 5 2600X Six-Core Processor (3600.10-MHz K8-class CPU) > (12 threads). > > FreeBSD 12.2-RELEASE-p7 r369865 GENERIC amd64 (SCHED_ULE) vs > FreeBSD 12.2-RELEASE-p7 r369865 M5P amd64 (SCHED_4BSD). > > Comparing "make buildworld" time with misc/dnetc running vs not > running. (misc/dnetc is your basic 100% compute-bound task, running > at nice 20.) > > Three out of the four combinations build in roughly four hours, but > SCHED_ULE with dnetc running takes close to twelve! (And that was > overnight with basically nothing else running.) This is an even > worse disparity than I have seen in previous releases. I do not use dnetc, but shed_ule on 2700 compile wold faster than 4 hours. With ccache it takes ~10 minutes: world+kernel build and install and update loaders. # Make an SMP-capable kernel by default options SMP #b Symmetric MultiProcessor Kernel options NUMA #o Non-Uniform Memory Architecture support options EARLY_AP_STARTUP #o device cpufreq #m for non-ACPI CPU frequency control device cpuctl #m Provides access to MSRs, CPUID info and microcode update feature. # Kernel base options SCHED_ULE #b 4BSD/ULE scheduler options _KPOSIX_PRIORITY_SCHEDULING #b POSIX P1003_1B real-time extensions options PREEMPTION #b Enable kernel thread preemption and sysctl tunings on desktop only: # SCHEDULER kern.sched.steal_thresh=1 # Minimum load on remote CPU before we'll steal // workaround for freezes kern.sched.balance=0 # Enables the long-term load balancer kern.sched.balance_interval=1000 # Average period in stathz ticks to run the long-term balancer kern.sched.affinity=10000 # Number of hz ticks to keep thread affinity for From eugen@grosbein.net Thu Jul 8 22:00:41 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 93299122B454; Thu, 8 Jul 2021 22:01:03 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLVby3sLMz3J7X; Thu, 8 Jul 2021 22:01:02 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 168M0pM2054510 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 8 Jul 2021 22:00:52 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-net@freebsd.org Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 168M0oQ6074015 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Fri, 9 Jul 2021 05:00:50 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: How to run script on network address change? To: Rozhuk Ivan , freebsd-net@freebsd.org, freebsd-hackers@FreeBSD.org References: <20210620165215.0f3dd15c@rimwks.local> From: Eugene Grosbein Message-ID: <886e5afe-7b4f-db1d-4d41-08a7457c6bdc@grosbein.net> Date: Fri, 9 Jul 2021 05:00:41 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <20210620165215.0f3dd15c@rimwks.local> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-102.3 required=5.0 tests=BAYES_00,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -100 SHORTCIRCUIT No description available. * [score: 0.0002] * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4GLVby3sLMz3J7X X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [0.89 / 15.00]; RCVD_TLS_ALL(0.00)[]; R_SPF_FAIL(1.00)[-all]; NEURAL_SPAM_SHORT(0.92)[0.925]; FREEFALL_USER(0.00)[eugen]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; DMARC_NA(0.00)[grosbein.net]; SPAMHAUS_ZRD(0.00)[2a01:4f8:c2c:26d8::2:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:4f8:c2c:26d8::2:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.93)[-0.932]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org,FreeBSD.org]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-hackers,freebsd-net] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N 20.06.2021 20:52, Rozhuk Ivan wrote > Hi! > > > I have FreeBSD 13@amd64 router with DHCP client on uplink. > > How to run script on IP address change? Here is my /etc/dhclient-enter-hooks I use with FreeBSD 11 and stock dhclient for a system with two distinct ISP uplinks. It replaces some of default hooks: #!/bin/sh # Ignore ISP DNS servers as we run our own resolver add_new_resolv_conf() { return 0 } add_new_address() { # Perform interface configuration eval "$IFCONFIG $interface \ inet $new_ip_address \ netmask $new_subnet_mask \ broadcast $new_broadcast_address \ $medium" # Save ISP gateway address with interface description for easy access eval '$IFCONFIG $interface description "gw $new_routers"' # Generate nice logs $LOGGER "New IP Address ($interface): $new_ip_address" $LOGGER "New Subnet Mask ($interface): $new_subnet_mask" $LOGGER "New Broadcast Address ($interface): $new_broadcast_address" $LOGGER "New Routers ($interface): $new_routers" # Some additional logic may be added here like switching default route return 0 } # EOF From nobody Sat Jul 10 13:54:17 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B66361241A68 for ; Sat, 10 Jul 2021 13:54:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMWjh69tlz3MYF; Sat, 10 Jul 2021 13:54:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 16ADsHmU048955 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 10 Jul 2021 16:54:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 16ADsHmU048955 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 16ADsHL4048954; Sat, 10 Jul 2021 16:54:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 10 Jul 2021 16:54:17 +0300 From: Konstantin Belousov To: Vitaliy Gusev Cc: freebsd-hackers@freebsd.org, gljennjohn@gmail.com, Mark Johnston , Peter Grehan Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Message-ID: References: <20210703065420.6dbafb5f@ernst.home> <2390FA9B-319E-45D4-BEA7-10878E43AD4B@gmail.com> <2A7A1E60-766D-427B-8288-E92B4090022E@gmail.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GMWjh69tlz3MYF X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [1.70 / 15.00]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCPT_COUNT_FIVE(0.00)[5]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; R_DKIM_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.29)[-0.291]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-0.01)[-0.011]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; SUBJECT_ENDS_QUESTION(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N On Tue, Jul 06, 2021 at 01:21:17AM +0300, Vitaliy Gusev wrote: > Comments are below, > > > commit 0392eb3c93b7dacc31dbdf8ec2fc40fa5ba67c62 > > Author: Konstantin Belousov > > > Date: Mon Jul 5 21:53:22 2021 +0300 > > > > madvise(MADV_FREE): try harder to handle shadow chain > > > > In particular, collapse top object and see if there is no backing object > > after, which means that we would not revert to older content if drop the > > top object. > > > > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > > index 1ac4ccf72f11..80abac223f29 100644 > > --- a/sys/vm/vm_map.c > > +++ b/sys/vm/vm_map.c > > @@ -3033,6 +3033,7 @@ vm_map_madvise( > > entry = vm_map_entry_succ(entry); > > for (; entry->start < end; > > entry = vm_map_entry_succ(entry)) { > > + vm_object_t obj; > > vm_offset_t useEnd, useStart; > > > > if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) != 0) > > @@ -3046,9 +3047,16 @@ vm_map_madvise( > > * backing object can change. > > */ > > if (behav == MADV_FREE && > > - entry->object.vm_object != NULL && > > - entry->object.vm_object->backing_object != NULL) > > - continue; > > + (obj = entry->object.vm_object) != NULL && > > + obj->backing_object != NULL) { > > + VM_OBJECT_WLOCK(obj); > > + if ((obj->flags & OBJ_DEAD) != 0) > > + continue; > > Here is object is left locked, however I didn’t met this condition. Right. > > > + vm_object_collapse(obj); > > + VM_OBJECT_WUNLOCK(obj); > > + if (obj->backing_object != NULL) > > + continue; > > After testing it looks that backing_object is not null here, memory is not freed and mmapfork test program is still killed. Any ideas? > I finally looked at this thing again. You do madvise(PAGE_SIZE). With my patch (updated version below) issue madvise() for the whole region once. commit b183ec9e985961bd0b450d9e7bdbbcde751c830b Author: Konstantin Belousov Date: Mon Jul 5 21:53:22 2021 +0300 madvise(MADV_FREE): try harder to handle shadow chain In particular, collapse top object and see if there is no backing object after, which means that we would not revert to older content if drop the top object. diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index 1ac4ccf72f11..188b37560e0e 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -3033,6 +3033,7 @@ vm_map_madvise( entry = vm_map_entry_succ(entry); for (; entry->start < end; entry = vm_map_entry_succ(entry)) { + vm_object_t obj; vm_offset_t useEnd, useStart; if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) != 0) @@ -3046,9 +3047,18 @@ vm_map_madvise( * backing object can change. */ if (behav == MADV_FREE && - entry->object.vm_object != NULL && - entry->object.vm_object->backing_object != NULL) - continue; + (obj = entry->object.vm_object) != NULL && + obj->backing_object != NULL) { + VM_OBJECT_WLOCK(obj); + if ((obj->flags & OBJ_DEAD) != 0) { + VM_OBJECT_WUNLOCK(obj); + continue; + } + vm_object_collapse(obj); + VM_OBJECT_WUNLOCK(obj); + if (obj->backing_object != NULL) + continue; + } pstart = OFF_TO_IDX(entry->offset); pend = pstart + atop(entry->end - entry->start); From nobody Sun Jul 11 20:01:22 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3341F12774BE for ; Sun, 11 Jul 2021 20:01:24 +0000 (UTC) (envelope-from leres@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNHpX15sBz3CH6 for ; Sun, 11 Jul 2021 20:01:24 +0000 (UTC) (envelope-from leres@freebsd.org) Received: from ice.alameda.xse.com (unknown [IPv6:2600:1700:a570:e20:f2ad:4eff:fe09:150e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: leres) by smtp.freebsd.org (Postfix) with ESMTPSA id D6A3C5ED6 for ; Sun, 11 Jul 2021 20:01:23 +0000 (UTC) (envelope-from leres@freebsd.org) From: Craig Leres To: freebsd-hackers@freebsd.org Subject: Did something change with sys/cpuset.h in CURRENT? Message-ID: Date: Sun, 11 Jul 2021 13:01:22 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N I've gotten a couple of pkg-fallout reports for security/zeek but I'm not finding any mailing list chatter referencing cpusetid_t. I also did a checkout of https://git.FreeBSD.org/src.git and don't see any recent changes (or anything in UPDATING) that would impact the use of sys/cpuset.h. Could someone please point me in the right direction? Craig Maintainer: leres@FreeBSD.org Log URL: http://beefy17.nyi.freebsd.org/data/main-i386-default/pf44e1c1de734_s63ca9ea4f3/logs/zeek-4.0.3.log Build URL: http://beefy17.nyi.freebsd.org/build.html?mastername=main-i386-default&build=pf44e1c1de734_s63ca9ea4f3 Log: =>> Building security/zeek build started at Sun Jul 11 05:19:11 UTC 2021 port directory: /usr/ports/security/zeek package name: zeek-4.0.3 building for: FreeBSD main-i386-default-job-05 14.0-CURRENT FreeBSD 14.0-CURRENT 1400025 i386 maintained by: leres@FreeBSD.org Makefile ident: Poudriere version: 3.2.8-8-gaf08dbda Host OSVERSION: 1400008 Jail OSVERSION: 1400025 Job Id: 05 [...] In file included from /wrkdirs/usr/ports/security/zeek/work/zeek-4.0.3/auxil/highwayhash/highwayhash/os_specific.cc:56: /usr/include/sys/cpuset.h:153:12: error: unknown type name 'cpusetid_t' int cpuset(cpusetid_t *); ^ /usr/include/sys/cpuset.h:154:18: error: unknown type name 'cpuwhich_t'; did you mean '__cpuwhich_t'? int cpuset_setid(cpuwhich_t, id_t, cpusetid_t); ^ /usr/include/sys/_types.h:146:14: note: '__cpuwhich_t' declared here typedef int __cpuwhich_t; /* which parameter for cpuset. */ ^ From nobody Sun Jul 11 22:19:49 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 738EB8D7C09 for ; Sun, 11 Jul 2021 22:20:02 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-ua1-x934.google.com (mail-ua1-x934.google.com [IPv6:2607:f8b0:4864:20::934]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNLtV2jnKz3sDw; Sun, 11 Jul 2021 22:20:02 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: by mail-ua1-x934.google.com with SMTP id d2so6384216uan.0; Sun, 11 Jul 2021 15:20:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=RAnQAKvyK5M7eccKr2F4bACeBqQvh3+RNLrQlzMuuws=; b=RZAFYKQAqMYNQXWNvXtD87znwjyAwifQmum17jhepJzaQLwMUOZs+1abuvbWx7zgb6 lZ3OBZOhSaWQF5y2JIyx59BBzSDT8NCpK6Ymyje31ny8nNfz5vezl+3cDcDcFWW4CtVX PJo5INsM26ZW1gaaXu42tQwFP6OlhHhHLdFudyi2d26vV+FiLcuj1HxrnKz5pZxH4ZR7 pvWUAJXUPm41S4cCDtiINqkxvLMvaHSxt8BEGlwCWCdOludOLhSeqVCNQPnw3YlKJvcc v6sRVFmT1EnS/ncHbVpkVlQC07I6jWwn4Tv6q6hZ5bGhj0yZl+YxcmyXE39nHb7D2V1E tsXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=RAnQAKvyK5M7eccKr2F4bACeBqQvh3+RNLrQlzMuuws=; b=jPCkBJ9Z04xFq6cb7MgBI0+B4AsS2p1DIiSVNA57k0AzTdXTurB76U3emALiP1cgNT A8ZgRYSGk4NoDo5VYrPupfbaPL7uiKjekxApeQlsqrBuWas3SrvbOXgtdApNjnSWcPeN MUmQwrl9RpWH5SQgb+GXJ5YMGbxUaNB3tF5UipCzQ+PAHCXqGT+U6OPcPCRBTReDTzLg MXZc0qwgOG97HbTCe5/D/Lnl9PbI6F9s4j6mfy7ythPlgzc0Zp2ZIonOc3tYV13p25Uu OtdgJM7E9dy29XC0xPsi0MBuxuvOUys4LX/G50o/ZtDBmeOEzdC9NOSEZdhlMrBaOSTy E2ew== X-Gm-Message-State: AOAM531Nq8fn6mo+m3N/Xbq0mDdHnewK0ufdKC5Xsh7UxFTs9C5KKD4W JpvoeSx1KOzkb/HyzdafrvMZRwn6JyL9+tKLye7K1RXEs6Q= X-Google-Smtp-Source: ABdhPJwB1gCYUBrcvM45gJMx/Dnvx5qo15MEQfadoWbIbFMrJ0IBt5IdvXNn/WrB99M/NMJzSikVowNjZWQpeWS/jEI= X-Received: by 2002:ab0:778c:: with SMTP id x12mr47677808uar.88.1626042001148; Sun, 11 Jul 2021 15:20:01 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Sergey Kandaurov Date: Mon, 12 Jul 2021 01:19:49 +0300 Message-ID: Subject: Re: Did something change with sys/cpuset.h in CURRENT? To: Craig Leres Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000de502905c6e06706" X-Rspamd-Queue-Id: 4GNLtV2jnKz3sDw X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: Y --000000000000de502905c6e06706 Content-Type: text/plain; charset="UTF-8" On Sun, 11 Jul 2021 at 23:02, Craig Leres wrote: > > I've gotten a couple of pkg-fallout reports for security/zeek but I'm > not finding any mailing list chatter referencing cpusetid_t. I also did > a checkout of https://git.FreeBSD.org/src.git and don't see any recent > changes (or anything in UPDATING) that would impact the use of > sys/cpuset.h. > > Could someone please point me in the right direction? > > Craig > > Maintainer: leres@FreeBSD.org > Log URL: > > http://beefy17.nyi.freebsd.org/data/main-i386-default/pf44e1c1de734_s63ca9ea4f3/logs/zeek-4.0.3.log > Build URL: > > http://beefy17.nyi.freebsd.org/build.html?mastername=main-i386-default&build=pf44e1c1de734_s63ca9ea4f3 > Log: > > =>> Building security/zeek > build started at Sun Jul 11 05:19:11 UTC 2021 > port directory: /usr/ports/security/zeek > package name: zeek-4.0.3 > building for: FreeBSD main-i386-default-job-05 14.0-CURRENT FreeBSD > 14.0-CURRENT 1400025 i386 > maintained by: leres@FreeBSD.org > Makefile ident: > Poudriere version: 3.2.8-8-gaf08dbda > Host OSVERSION: 1400008 > Jail OSVERSION: 1400025 > Job Id: 05 > > [...] > > In file included from > > /wrkdirs/usr/ports/security/zeek/work/zeek-4.0.3/auxil/highwayhash/highwayhash/os_specific.cc:56: > /usr/include/sys/cpuset.h:153:12: error: unknown type name 'cpusetid_t' > int cpuset(cpusetid_t *); > ^ > /usr/include/sys/cpuset.h:154:18: error: unknown type name 'cpuwhich_t'; > did you mean '__cpuwhich_t'? > int cpuset_setid(cpuwhich_t, id_t, cpusetid_t); > ^ > /usr/include/sys/_types.h:146:14: note: '__cpuwhich_t' declared here > typedef int __cpuwhich_t; /* which parameter for cpuset. */ > ^ > In zeek-4.0.3 (to which the port was recently updated from 4.0.2) the header inclusion order has changed; the relevant diff: #ifdef __FreeBSD__ #define OS_FREEBSD 1 -#include #include +#include #include #else #define OS_FREEBSD 0 - sys/cpuset.h requires sys/types.h to define cpusetid_t (and friends), - sys/params.h includes sys/types.h this is documented in cpuset(2) This works on stable versions such as FreeBSD 13.0 w/ clang 11.0.1, though. This is seemingly due to a header pollution fixed in llvmorg-12 import. Before that, sys/types were included indirectly with #include : random -> istream -> ostream -> locale -> nl_types.h -> sys/types.h With llvmorg-12 import, istream / ostream are replaced with iosfwd. --000000000000de502905c6e06706--