From nobody Tue Feb 27 05:48:29 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TkRPF0jWNz5CW8V for ; Tue, 27 Feb 2024 05:48:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-22.consmr.mail.gq1.yahoo.com (sonic317-22.consmr.mail.gq1.yahoo.com [98.137.66.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TkRPC3KFrz4M4C for ; Tue, 27 Feb 2024 05:48:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=s+cGj1zR; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1709012923; bh=uVXGLF5N0AHKv2im6hT5MdheqBTmjuYCWpQGu958xtw=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=s+cGj1zRIKcaxmo/PsJGB/jnYlzdKxzTOkxU4cPvhIKv7yyL4zv1oa5JDHxUfqO9pf63dUmy8IHGMlwTEE/LVjQwXGGmm1ExHNhquQuGtd5NzL5sZuUvJ+oePaI1AImqaVICl3idfwZrlG3+GiRUZtPwAZe2mnxTxyLk83emMmbHW69NfboMjafPnI/GUJ53788MclKY2HwbyEuUurOZM17WYJfKcmiA05L46X+B1kiJneD64X3bF0b5VR94qcnO7uHamTXacXt+jmQUwGNb5rXozYLaxsJ9JZKN2V8WRKM4RnHNmOFTixbT9xBP/YO3IuCKeDZzAoZCOO4QcSBDYg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1709012923; bh=39oaBlRT/3DxohMOj+xeURiKkt4341PuNz6yFZbYpOa=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=kGDI7S0U0MbsUKIXcC4eC1PL00HAX6NOEaZGIMpgGyrmvXUkkyDm/U3gpaCYfI6GSRI98mflBnwWcxzyJGaJ+y4lfZ5zTA6Nfhcn167tZgtWT456SD/ppKWpLaOD2wDcz5S4hnjf4kBPir1swWxCxjCqhxJlXNNHdE2TPil0OUsCOXIXFxddh+QjySqGbj85t5FR1nEZY4LX7qXpIVuPSphNNplSzgqRhfS8Mw8Vby/Tq9JjgLnZeh+6vlKJetFWtdSTxTZ5FBZqwV8C83wr6WWs/OCsMfGO4h59p8y5J78huMly9jo7FJ4WOAPTccPGe1fr38Nz8cl4I3AajIS7zw== X-YMail-OSG: kMR9xeIVM1mh8JlqG8yKEx1x39i5WBbQcJIRlfytpoqphbrGNPKUiE4kpItFhyq g35j4Hpj..p2dfcr9Bop9PlVT_xJJZIQEpGjA7XBolUXu6YewJ.UFERWlxzldsYDTryjHYyeASVx EXGpbF76m..I63SaPrC_34iGdr7aW45F0Y0pwVheS01PuUlT9p3jiucGPnMUPY1mbvIWxA7cD2VH n7Kt3Cj5EQl0UpeBUAX4CAd5GBXkS4d2NLV_MVaQz6.qQMaG0lcD_zQCshYdYJpAvav0Al3aqO2P 6gSGYCold2TvvLdnmAYcxRyoX_Z6IDkb8KcRgicXIzmNxvmhMp7RzrsAwLEZAbFEIkS8LUAJlzdp xAfuggAWyW5bcJwdHTLC4sqBA13uV4nB92eyFuPgJ0nrj_NTk1mtVxbw1V_XB7KiQ5JmqlyK8hEq BpbzxLl3XiFRMEijHIq3UWmS0zgB2T5BzCDxX69PiY0mBLFiAHhUolZNAirlAanBE0DQXEAqolY1 noyfGO4wz89u155voLdLwLBqSyxXnEtr3nnTpICdhiTA088GAJsuLTxnCsD2LUYiGjaPB2pGOG6I vQVtRjE7NRpHR09gtqicf8oBLlQL7Pw1HgmBINaneKYmmU6K8w7a7lAprQgfITkzQOymIJRZC.aN e6ow2OPzCCEsjjcxu3kJg8M_hJKu6Nt0M8y3OA4TJVrZBim1DVx9wLhFtdlec58naTjDTy1aAyM7 9Dp1DhlygJGpOiC3xM_HHr6FYN8RC_TvJg3o83ka.KicWzcfApWUWeHOG945p1CDJytd8zi.MZ0J j0w5qla0oun6OPLP1QBDEaqKpwERj91Zcr1mGafJssnm7XtzS5TAVGuYBkUSJnB8yM4el0xaVi7Y NBtO9kPrPPdIglpdP0yoKz9RSjJTDRJlnBiqEB9_kEuQswp1z8k.HOTUDHeJjBMgYqoLFJ0PBNyv y6CcK5soVdlPmGHW.6.l3S_zOGpMJQ_VDUECCDZ1D.W52BuGbeMfNTKcNSx8tEFbEgh5X1OGzftw tfXSjxlBGFKm72cLMxNLY7FDE12lBHCGjqiAnN37jQSX7rZCzNcTtElPKBH2G57hR7kMWwtviV_G tmQ0xt1Qu6CyFcj2e5EgdZ0kvUef6UnvMDEokrJXw881uoGZna3QHP88NP3yO66r260AJgNm8qaE W5ys5wkwJOY.CETBBHLKGjUX3iV8luJBC777dhPqybyto_ZZME.jPn0fIi8u_54QwajrwhH65rNi 4I_tfXR5UAeIAgJSE4P9cJewXEdhZN1ZJ11XLcRaSAZStr5dlDmfEUQabhMbr2V7VbOIp_yQpPh9 .hkwYdm1C3_VR5ZTyHpN.K4CAg4Nqiv4uOFxFI.beK2gtKOcbzZmQudidJFjI.8OSHEGyQCZDM2N eX4XbcHPpfsKpN.kbPP5ES68deKLeUdSgkAtPinLh_Li.T8.aVwCFoEzJeCVQLRHK5id_dIxuWQd cxkVP6hlgYuX.n6pootE15PtqYwUKY4sDdvbI4iZLpeXSTaXGKm4pc0Oq_IZPECb0ELWEg057kYQ BLs._S9m.0rXzQh7wkd.mEpmmkt5VXkZH4KV_t3ZgteKxgmo2yEZvA5.7Ku2tTM14l_ieykHdOFY 9_2auyQN_t2mY27C7jiC5xbyHK_6swcHdB9_qq1m.5I01a5.87AiLopkrAOa_25xvXqzZark.UJg xe9HGhaKGNXgCYYa76E_ZlfVv08kj9k60QhkZQgSqOwcD6REz0eVLuE437yx8vuJhGp2VrDwA19J b7F34v1CiVZPSet51fjUdJR0LdwzRcKvmT07K1Lc9vGbHVnidEuaeoCMUQiY6EyuyjV5uP2miZII QDOAe0sUacjZdxt9eHADrANdJNKe3_fcWpC4gTYfBrBr6STF9QOd.VmNTQo._klG_PD7xC_51lbT k08jXVuMVEQemie9YVUqT.5GAxdJTATjMaHwbKurofV88M2PF76A910zPCLLJx73j8fVbW3Fa4F1 kM8zC.Nh7Nyf3PZopD7Ec5LMjvWweOFd7Ia3jB8CceuUnAJJex40CuPDAeaHb2QOkkY9S6U3xqAb x6tHoqEuJOIKn0lJvIbbz.QQEB_36x4v154avFjRowl2hSOOsbJirnUOJXvjOW06sGY0j.6l1zy9 NilvOI65a8XZvIgFTj50Tb36G019.E8ULL9AK59F7wkOBbX4gzneWB.X0j7op5N11xerD3GbrxgJ mBvPQHbjebMjK__TvN6P2Quty2HkZaJS4LyzeB2VXdJ8NyotDX_Ws9Xu2gFlcL13y2HWr0N5j3bs CvOb3 X-Sonic-MF: X-Sonic-ID: 6a10d858-4d85-4e7d-a17b-2982201dc2dc Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Tue, 27 Feb 2024 05:48:43 +0000 Received: by hermes--production-gq1-5c57879fdf-hjdnf (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 28662a6d25ef6001c0d8fd5d4466a3ef; Tue, 27 Feb 2024 05:48:40 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: 13-STABLE high idprio load gives poor responsiveness and excessive CPU time per task Message-Id: <4D5E09B1-EA6A-4966-838D-0D003F997953@yahoo.com> Date: Mon, 26 Feb 2024 21:48:29 -0800 To: "Edward Sanford Sutton, III" , FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3774.400.31) References: <4D5E09B1-EA6A-4966-838D-0D003F997953.ref@yahoo.com> X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; SUBJECT_ENDS_SPACES(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FREEMAIL_TO(0.00)[hotmail.com,freebsd.org]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.148:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.148:from] X-Rspamd-Queue-Id: 4TkRPC3KFrz4M4C Questions include (generic list for reference, even if some has been specified): For /boot/loader.conf (for example) : What value of sysctl vm.pageout_oom_seq is in use? This indirectly adjusts the delay before sustained low free RAM leads to killing processes. Default 12 but 120 is what I use across a wide variety of systems. More is possible. For /etc/sysctl.conf : What values of sysctl vm.swap_enabled and sysctl vm.swap_idle_enabled are in use? (They work as a pair.) Together they can avoid kernel stacks beings swapped out. (Processes still can page out inactive pages, but not their kernel stacks.) Processes withe their kernel stacks swapped out to storage media do not run until the kernel stacks are swapped back in. Avoiding such for kernel stacks of processes involved in interacting with the system can be important ot maintaining control. This is a big hammer that is not limited to such processes. Both being 0 is what leads to kernel stacks not being swapped out. For /usr/local/etc/poudriere.conf : What values of the following are in use? NO_ZFS USE_TMPFS PARALLEL_JOBS ALLOW_MAKE_JOBS MAX_EXECUTION_TIME NOHANG_TIME MAX_EXECUTION_TIME_EXTRACT MAX_EXECUTION_TIME_INSTALL MAX_EXECUTION_TIME_PACKAGE MAX_EXECUTION_TIME_DEINSTALL (Some, of course, may still have the default value so the default value would be the answer in such cases.) Also: Other system tmpfs use outside poudriere? ZFS in use in system even if poudriere has NO_ZFS set? (Such is likely uncommon but is possible.) (Other contexts than poudriere could have some analogous questions.) For /usr/local/etc/poudriere.d/make.conf (for example) : What value of the likes of MAKE_JOBS_NUMBER is in use. Note: PARALLEL_JOBS, ALLOW_MAKE_JOBS, and the likes of MAKE_JOBS_NUMBER has as context the number of hardware threads in the context. The 3 load averages (over different time frames) vs. the hardware threads for the system is relevant information. Note: with various examples of package builds that use 25+ GiBytes of temporary file space, USE_TMPFS can be highly relevant, as is the RAM space, SWAP space, and the resultant RAM+SWAP space. But just the file I/O can be relevant, even if there is no tmpfs use. There are questions like: Spinning rust media usage? (An over-specific but suggestive reference form the more general subject area.) Serial console shows a responsiveness problem? Simple ssh session over local EtherNet? Only if there is a GUI present, even it is not being actively used? Only GUI interactions show a responsiveness problem? Going in another direction . . . I'm no ZFS tuning expert but I had performance problems that I described on the lists and the person that had increased vfs.zfs.per_txg_dirty_frees_percent had me try setting it back to vfs.zfs.per_txg_dirty_frees_percent=5 . In my context, the change was very helpful --but, to me, it was pure magic. My point is more that you may need judgments from someone with appropriate internal ZFS knowledge if you are to explore tuning ZFS. I've no evidence that the specific setting would be helpful. There has been a effort to deal with arc_prune problems/overhead. See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275594 === Mark Millard marklmi at yahoo.com