From owner-freebsd-hackers@freebsd.org Sun Mar 28 00:03:41 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5EF315ACDA2 for ; Sun, 28 Mar 2021 00:03:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7GC02Xvhz4sBm for ; Sun, 28 Mar 2021 00:03:40 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x729.google.com with SMTP id q3so9030949qkq.12 for ; Sat, 27 Mar 2021 17:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:subject:message-id:mime-version :content-disposition; bh=MsWly9YKpnslp1L7NbZQaoCylYRMfrgxIHQKG7InyMs=; b=tET/RcImrciQbrZ9/m8/UwV9Zwp8elLYcluPqXf/IbCHTnd+n2gSnbiwJxsRXzw0La 77XV7q6Svhq5+AcljihVgGSLTBTLKcSzvvBqeay8Ghd/RcFVEYcyLrKUrVQ9WOeHFN18 FDX2m9nfwt8ZlwWY1+ZZSKHjnT6GaGjGwXp9gyjJShVvbbfd0Hi0b0tSGNDcxbV+WOH4 c6QeAtiQ5PmpWXP7VpCr09WSx3gCq6aK38xOkHsBNX6CNKK/XVWwnj+wSmhJt7QZN/n1 AyaXPX9tkIy6oLGbHN/3rEiKI3iukAGm0LWkBwIy3wLrZQW1MZY3eh4h8btnbEd/Bpr7 rtdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:subject:message-id :mime-version:content-disposition; bh=MsWly9YKpnslp1L7NbZQaoCylYRMfrgxIHQKG7InyMs=; b=icbLAJM67OAx1Th9km7vGhi8zwnK5X6g+llIDtL6VmSPvQA8TsMHeqrDbkBaTAPpfR kuCK3Ir+u65viAEpT39AwOAFsAge3MyTQb8RpKA5NkVwTUn8QAZUBtqpQU9+GvhpKKbm oUGf6Botr7XbxHjMGdQ6WmI0hwezBFz48XPZEi8AJh2SpPbFWLtU8OokyrxDsTyNkFqI D6LUw6h008bUNPgpqXiUCa3YOJdEAW5BD2GLvxqeXWeKwgDelx3Sctniv/bpmaxIyLlO JBdtJIqB10nL7PWOY9Mya0mE7nJiRVGDOsQrVSbxsQBXig9YBe6o5m/1Gtag2k7TyPXp AYRA== X-Gm-Message-State: AOAM531KGdpRKNpAvdxZ8h/SLpSRukayoqscKmUzWrMUIEVyavwbrMrk Gae2pJ4Ep6su9RQT+iJ2WPJUDVV4W4rfWg== X-Google-Smtp-Source: ABdhPJxZf/H2Jg4SaSx4kdCAGp2aYvJG2uIIKIj0RK+YIrewueaIZXBlZD8s2KKFD9zl1HvKTduIdA== X-Received: by 2002:a37:c13:: with SMTP id 19mr19111639qkm.210.1616889819275; Sat, 27 Mar 2021 17:03:39 -0700 (PDT) Received: from nuc ([142.126.164.150]) by smtp.gmail.com with ESMTPSA id w78sm10019536qkb.11.2021.03.27.17.03.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Mar 2021 17:03:38 -0700 (PDT) Sender: Mark Johnston Date: Sat, 27 Mar 2021 20:03:39 -0400 From: Mark Johnston To: freebsd-hackers@freebsd.org Subject: KASAN port Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 4F7GC02Xvhz4sBm X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=tET/RcIm; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::729 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.69 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.99)[-0.995]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::729:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::729:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::729:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 00:03:41 -0000 Hi, I ported the KASAN implementation from NetBSD to FreeBSD. This is a testing and debugging tool that leverages compiler instrumentation to maintain a kernel "shadow" map which stores information about which addresses in the main kernel are safe to access. If you've been paying attention to recent kernel commits you may have noticed that several bugs have been found and fixed using this tool already; there are several more that I'm aiming to have fixed in 13.0. There was a GSOC project by Costin Carabas and andrew@ which did an initial port of KASAN and several other debugging facilities; I reused a few pieces of that work but this was mostly a direct port. The instrumentation and validity checking introduces a fairly substantial performance hit. I think a 2-3x slowdown is pretty typical, but it could be more for workloads which execute a lot of kernel code. It's best used in conjunction with test suites that exercise a lot of kernel functionality, like the regression test suite, stress2 or syzkaller. KASAN is currently only implemented for amd64. It would be a useful and probably relatively small project to port it to platforms like arm64 and riscv. If anyone is interested in this, please contact me. I posted reviews for various pieces of the port here: https://reviews.freebsd.org/D29454: Add a KASAN option to the kernel build https://reviews.freebsd.org/D29416: Add the KASAN runtime https://reviews.freebsd.org/D29417: amd64: Implement a KASAN shadow map https://reviews.freebsd.org/D29455: amd64: Add MD bits for KASAN https://reviews.freebsd.org/D29456: uma: Add KASAN state transitions https://reviews.freebsd.org/D29457: kstack: Add KASAN state transitions https://reviews.freebsd.org/D29458: kmem: Add KASAN state transitions https://reviews.freebsd.org/D29459: vfs: Add KASAN state transitions for vnodes https://reviews.freebsd.org/D29460: execve: Mark exec argument buffers https://reviews.freebsd.org/D29461: malloc: Add state transitions for KASAN A couple of small LLVM changes are also required: https://reviews.llvm.org/D98285 https://reviews.llvm.org/D98286 Please ask questions and provide review feedback. To test the port, grab https://github.com/markjdb/freebsd/tree/ff/kasan and: $ make kernel-toolchain WITHOUT_SYSTEM_COMPILER= $ make buildkernel KERNCONF=GENERIC-KASAN There are a few limitations of the current implementation, especially from the fact that we don't have a way to disable all uses of the direct map. However, we have a way to reduce usage of that map by kernel memory allocators and that's enough to find non-trivial bugs, so it seems worthwhile to commit it now and continue to refine it. From owner-freebsd-hackers@freebsd.org Sun Mar 28 00:23:14 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D39C45ADE20 for ; Sun, 28 Mar 2021 00:23:14 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7GdY6z00z4tRc for ; Sun, 28 Mar 2021 00:23:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qt1-x82d.google.com with SMTP id s2so6934431qtx.10 for ; Sat, 27 Mar 2021 17:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:subject:message-id:mime-version :content-disposition; bh=ZFSVqxy4+lssU/YnffkL7TS95eKWWpIIx8Xd2Y4SkXc=; b=l9E4CA2ujYHNFi391z0hJAOZws6gNfrUClgi9lvKWXyJ6XXHcsHLzu1CAFh12wuEHS bO4avGpEn4s5XuP11dNGKUlULAGU6326cuX922WZ1ejguLSNjjfxi1V0E843DuMlPxuj MqxnjsOmtXwml+A3f+ORAULXtiS+1kcXsAjoA99Sj+HTrLLst4qivHzP0RKSxH0xjfUi NOKsbWJkl/ki5siLKStqQJ9pSIKeWWgLa+wkOeU2Dha849FzyKt/gWmuO68jF79Q7yT1 334WCUOQP5PUYzCfrAu+4E5gTFtV+FCPBYNcLTwLD/mztvxPWJ3Q2ayz7A5awzyNBiUy jCNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:subject:message-id :mime-version:content-disposition; bh=ZFSVqxy4+lssU/YnffkL7TS95eKWWpIIx8Xd2Y4SkXc=; b=rsv4mvXY+FCK1kZEdw7NUT4vpE3AD8z/Q7ZrlSgga2kZ1yHYbW3fWNPB3ugn7dVH1F vh2nYjPrgs591ZYXd3SZOPIT5TGn+nZhGpXzF958YB0pPtnxmhAiok4pe7HDHZSG8pR6 WRr8+7OD3ArZpIaqAYE/Pr+ckPtgQHkj8S4sGCZzCfcO//I81N7GFKPiTf+YTKRn6BNH 1MWyOpeK6zQX78LR3WEP4yq09n4IOIVmRnKcLgIWyG6SawbBFgLTIV6zBwKz7ciIaRno Lo2PaqLW8K6RLWMPSsU/oeK4av2PxnKlUXNq6xt5mjEbZC6QtitJRCKZhr/vA3t03b5U Qa1g== X-Gm-Message-State: AOAM531q+Xh4MU/YlndCad21Sc82fJJLt03bEEezPDeUrGnt9nNCAfMf sTNKfVySmfZuBWcuxx+matRJ5tX5vJW1fA== X-Google-Smtp-Source: ABdhPJwmAXD4b8GyoOWTeTwb05vXdLsSMAWFoeIsQ3BDSHCvPjE2nOBjyNY/C+007FMC1fP0oQlfBA== X-Received: by 2002:aed:2981:: with SMTP id o1mr17300042qtd.386.1616890992858; Sat, 27 Mar 2021 17:23:12 -0700 (PDT) Received: from nuc ([142.126.164.150]) by smtp.gmail.com with ESMTPSA id c5sm9758692qkl.21.2021.03.27.17.23.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Mar 2021 17:23:12 -0700 (PDT) Sender: Mark Johnston Date: Sat, 27 Mar 2021 20:23:13 -0400 From: Mark Johnston To: freebsd-hackers@freebsd.org Subject: statement on FreeBSD development processes Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 4F7GdY6z00z4tRc X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=l9E4CA2u; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::82d as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.998]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82d:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82d:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82d:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 00:23:14 -0000 Dear FreeBSD Community, In light of the recent commentary on FreeBSD's development practices, members of the Core team would like to issue the following statement. Code quality is an essential FreeBSD value: From the 1980s when work on BSD became the de facto standard TCP/IP stack, to our more recent work around performance scalability on multicore, attention to detail is critical. The recent concerns regarding the WireGuard patches remind us that our development processes must always continue to mature. While the project has historically, and aggressively, led the way in adopting new development methodologies - from public version control to being early adopters of static analysis tools such as Coverity - these events have brought to light a real gap that needs to be addressed. The high stability and quality of FreeBSD is a testimony to the experience of our developers. As in any open source project, we rely on developers to exercise good judgement in seeking review and committing new features, and to follow the guidelines laid out in the Committer's Guide. We make heavy use of public code review, and FreeBSD developers spend a significant amount of time improving each others' contributions. We were excited to provide a kernel WireGuard implementation in FreeBSD 13.0. Before the if_wg(4) rewrite was committed, several FreeBSD developers proactively worked on fixing bugs and writing tests and documentation for the original implementation. In other words, we had spent time during the release's Q/A period looking for problems, and that unfortunately culminated in if_wg(4) being removed from 13.0 during the release cycle. As FreeBSD developers, it is incumbent on each of us to support each other's work by providing code review and helping test and fix the code. This incident highlights the need to do this work more proactively, and to maintain a robust, multi-layered development process that can catch problems as they fall through the cracks. Over the next month the FreeBSD Core Team will lead a discussion on appropriate pre-commit testing, static analysis, code review, and integration policies to avoid a repeat of this situation and to continue improving FreeBSD's code quality. We know there will be challenges in key areas, such as third-party device drivers, and components of the system where fewer developers have sufficient expertise. The FreeBSD Foundation has full-time staff members participating in significant code review today, and is committed to supporting the needs identified by the Core team and the developer community for this effort. We look forward to input from the community on our proposals for updated policies as we move forward, maintaining high code quality as a core value for FreeBSD. Thanks, -Mark, with core@ hat on From owner-freebsd-hackers@freebsd.org Sun Mar 28 16:31:39 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0F7B357EB2F for ; Sun, 28 Mar 2021 16:31:39 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F7h6y67z6z4pQB for ; Sun, 28 Mar 2021 16:31:38 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.nyi.freebsd.org (Postfix) id D2CA257EAD4; Sun, 28 Mar 2021 16:31:38 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D28FF57EB2E for ; Sun, 28 Mar 2021 16:31:38 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [107.204.234.170]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7h6x4fQPz4pRW for ; Sun, 28 Mar 2021 16:31:37 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.16.1/8.15.2) with ESMTP id 12SGVauQ002056 for ; Sun, 28 Mar 2021 16:31:36 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.16.1/8.16.1/Submit) id 12SGVa3V002055 for hackers@freebsd.org; Sun, 28 Mar 2021 09:31:36 -0700 (PDT) (envelope-from david) Date: Sun, 28 Mar 2021 09:31:36 -0700 From: David Wolfskill To: hackers@freebsd.org Subject: Experiments with suspend/resume (amd64; nvidia-driver) Message-ID: Reply-To: hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="btpbIvfe0ckCnltw" Content-Disposition: inline X-Rspamd-Queue-Id: 4F7h6x4fQPz4pRW X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of david@catwhisker.org designates 107.204.234.170 as permitted sender) smtp.mailfrom=david@catwhisker.org X-Spamd-Result: default: False [-0.12 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[hackers@freebsd.org]; FREEFALL_USER(0.00)[david]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[107.204.234.170:from]; R_SPF_ALLOW(-0.20)[+ip4:107.204.234.170:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[107.204.234.170:from:127.0.2.255]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.72)[-0.715]; DMARC_NA(0.00)[catwhisker.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:7018, ipnet:107.192.0.0/12, country:US]; REPLYTO_EQ_TO_ADDR(5.00)[]; MAILMAN_DEST(0.00)[hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 16:31:39 -0000 --btpbIvfe0ckCnltw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I am trying to get a laptop running stable/13 working at least as well as it does under stable/12. At the moment, stable/13 has the advantage that the mouse works (under stable/12, it does not), but the "resume" part of "suspend/resume" works under stable/12, but does not under stable/13. For the specific points at which the behaviors are noted, here are the uname strings in question, for reference: FreeBSD g1-48.catwhisker.org 12.2-STABLE FreeBSD 12.2-STABLE #985 stable/12= -n232904-6008a5fad3c: Sun Mar 28 03:33:20 PDT 2021 root@g1-48.catwhiske= r.org:/common/S1/obj/usr/src/amd64.amd64/sys/CANARY amd64 1202505 1202505 FreeBSD g1-48.catwhisker.org 13.0-STABLE FreeBSD 13.0-STABLE #182 stable/13= -n245050-c7d10e7ec872: Sun Mar 28 04:27:46 PDT 2021 root@g1-48.catwhisk= er.org:/common/S3/obj/usr/src/amd64.amd64/sys/CANARY amd64 1300500 1300500 As I mention behavior under head, as well: FreeBSD g1-48.catwhisker.org 14.0-CURRENT FreeBSD 14.0-CURRENT #191 main-n2= 45702-1c1ff7979571: Sun Mar 28 04:02:23 PDT 2021 root@g1-48.catwhisker.= org:/common/S4/obj/usr/src/amd64.amd64/sys/CANARY amd64 1400006 1400006 For each of the environments, /etc/src.conf contains: PORTS_MODULES+=3Dx11/nvidia-driver and the x11/nvidia-driver modules are rebuilt whenever the kernel is. * Under stable/12, the machine is normally run with kern.vty=3D"sc" in /boot/loader.conf and (as noted) suspend/resume Just Works nearly all the time (based on my experiences with an older laptop that is similarly configured -- from back when I was still commuting to work, and thus had occasion to exercise the suspend/resume code). I note that this machine (Dell Precision 7520) does not cause any lights that I can see to "blink" while suspended -- unlike the Dell Precision M4800. A few minutes ago, I commented that line in /boot/loader.conf out and rebooted the machine and re-tested suspend/resume. I was gratified to note that suspend/resume still worked, even with vt. X11 worked, of course, but vtys are ... not useful for displaying human-readable information. (It looks like some wildly-colored character-cell-sized boxes.) I switched it back to sc. * Under stable/13, after the recent work in loader by tsoome@, I had switched to letting vty default to vt (just before stable/13 was branched, IIRC). Thus, recalling some earlier issues where vt appeared to be implicated in failure of "resume" to (re-?)initialize the video display, I had thought that the use of vt (vs. sc) might be at the root of the failure to resume for stable/13 (and head). However, an experiment showed that after an attempted resume, which casued the power light to come back on (while the screen stayed dark and the machine remained unresponsive to anything but the power button -- well, I suppose unplugging it and removing the battery would probably have caused a reaction, as well, but that's a bit extreme -- I was unable to do as much as ping the machine from another one. That seems to me to indicate something rather more profound than "video did not re-initialize." * Unsurprisingly, behavior under head is as described above for stable/13. How might I proceed to get the above resolved? While I use the M4800 quite heavily, the 7520 spends most of its time powered off, so making use of it seems like a Good Idea. :-) I have dumped some files from the machine at http://www.catwhisker.org/~david/FreeBSD/suspend_resume/; I am happy to supplement those. Peace, david --=20 David H. Wolfskill david@catwhisker.org Describing the 6 Jan 2021 Capitol insurrection as "hugging and kissing the police and the guards, you know, they had great relationships" is what I'd expect from someone who conflates affection with sexual assault. See https://www.catwhisker.org/~david/publickey.gpg for my public key. --btpbIvfe0ckCnltw Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEE4owz2QxMJyaxAefyQLJg+bY2PckFAmBgr2dfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEUy OEMzM0Q5MEM0QzI3MjZCMTAxRTdGMjQwQjI2MEY5QjYzNjNEQzkACgkQQLJg+bY2 PcmIbQf+KAHwHZ4J8lOQr68gj/xtd5fM8p+AOzRFw1x4x5iBrbsNDL6AdJA9iFSC 4CJIqc0y2tChJ7V87zspYuEpGHxDrYnTil+Or25rmFIQrHRKO5xWJVoiEQ7DdTWB qa+jXb5P1fW0+e3PFlYHo0ZZUyWvWAhq+RVhDT7uCH9WT6/sjEI7St/UeFJZb2oi S7UvRQ2L4C/tjwko890R37N59/cmwhwwva2oZA9zzYl/sZgxXr42vn8FF+g4Isee 1rgyAkslWAvlkbBJUal8kB3RGnSlhJ3hEcbuCIID3XiRuLR3wgNtoiWkU/gWjhWg 8kBCZurnbQDGy/WgwO+KueCbnx3ghA== =E/Mb -----END PGP SIGNATURE----- --btpbIvfe0ckCnltw-- From owner-freebsd-hackers@freebsd.org Tue Mar 30 05:06:51 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 97CE45BA6EC for ; Tue, 30 Mar 2021 05:06:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8cqv2xyGz3lps for ; Tue, 30 Mar 2021 05:06:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 637BF5BA6EB; Tue, 30 Mar 2021 05:06:51 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 634005BA6EA for ; Tue, 30 Mar 2021 05:06:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8cqt2yCCz3lmy for ; Tue, 30 Mar 2021 05:06:50 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f174.google.com with SMTP id c16so15327325oib.3 for ; Mon, 29 Mar 2021 22:06:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=l1iBiQUScXEFDF0S8Hdcc0JmhUFXPSD7OAhERUgOit8=; b=XfAz1B69YJRFH+5v3DWivxNmZRMGe8t9Wb/BTI1puriYtV4kVcyjGG80nIAPEtYxpH AS4RibZvZ1COvNAngkMfTsbVGiG06VYwBKZi67n5/1roA7CL8Qoy7dc1/u5RfXIxWJ9U CP9cxxIKtONZnT7rBHUJ9OYMmyQACpwStNvs9vCfyOdAmjYEwaJQ1ktwT9uMPTjhvJ1d 344MwGAlguGl/iemE2vVVAR+PMbCtvXt1sQGo0MSRF/1TUZjFQJY3wGTGzUAT5nGb8Gj iFrm+TugsOEDpcX5clL9pRdESIUi0sjGcqBXnVvi7pSPg27WsEZ5ClZDK8s/wpELuyf6 4R5w== X-Gm-Message-State: AOAM533PBrwki8nbPYQfxpyDoIa++D5y/zEIn3DywJ2aWh22K2bekumu E296EOYGGsKdexx3RO/PKiZcXqMAkA0907MOFMoH+1hbO8Q= X-Google-Smtp-Source: ABdhPJyOeaBS33b8RDkNLsQ7bt5hYX516alfRSWp+imXCZmzZUJ2OgalE12QtEhZCfXYLHM7/B93r3oJnogpQHVeAec= X-Received: by 2002:a05:6808:14c8:: with SMTP id f8mr1962547oiw.55.1617080807504; Mon, 29 Mar 2021 22:06:47 -0700 (PDT) MIME-Version: 1.0 From: Alan Somers Date: Mon, 29 Mar 2021 23:06:36 -0600 Message-ID: Subject: How does the stack's guard page work on amd64? To: "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 4F8cqt2yCCz3lmy X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.167.174:from]; TO_DOM_EQ_FROM_DOM(0.00)[]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[209.85.167.174:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.174:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.174:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 05:06:51 -0000 Rust tries to detect stack overflow and handles it differently than other segfaults, but it's currently broken on FreeBSD/amd64. I've got a patch that fixes the problem, but I would like someone to confirm my reasoning. It seems like FreeBSD's main thread stacks include a guard page at the bottom. However, when Rust tries to create its own guard page (by re-mmap()ping and mprotect()ing it), it seems like FreeBSD's guard page automatically moves up into the un-remapped region. At least, that's how it behaves, based on the addresses that segfault. Is that correct? For other threads, Rust doesn't try to remap the guard page, it just relies on the guard page created by libthr in _thr_stack_alloc. Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's stack overflow detection worked in 10.3. -Alan From owner-freebsd-hackers@freebsd.org Tue Mar 30 09:35:06 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 31AC05C0A4F for ; Tue, 30 Mar 2021 09:35:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8knP6JtMz4V2h for ; Tue, 30 Mar 2021 09:35:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id D6A285C0AC4; Tue, 30 Mar 2021 09:35:05 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D54C85C0E1F for ; Tue, 30 Mar 2021 09:35:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8knP3llcz4V4w; Tue, 30 Mar 2021 09:35:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 12U9YtXP007755 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 30 Mar 2021 12:34:58 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 12U9YtXP007755 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 12U9YtJ6007754; Tue, 30 Mar 2021 12:34:55 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 30 Mar 2021 12:34:55 +0300 From: Konstantin Belousov To: Alan Somers Cc: "freebsd-hackers@freebsd.org" Subject: Re: How does the stack's guard page work on amd64? Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4F8knP3llcz4V4w X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 09:35:06 -0000 On Mon, Mar 29, 2021 at 11:06:36PM -0600, Alan Somers wrote: > Rust tries to detect stack overflow and handles it differently than other > segfaults, but it's currently broken on FreeBSD/amd64. I've got a patch > that fixes the problem, but I would like someone to confirm my reasoning. > > It seems like FreeBSD's main thread stacks include a guard page at the > bottom. However, when Rust tries to create its own guard page (by > re-mmap()ping and mprotect()ing it), it seems like FreeBSD's guard page > automatically moves up into the un-remapped region. At least, that's how > it behaves, based on the addresses that segfault. Is that correct? Show the facts. For instance, procstat -v (and a note which mapping was established by runtime for the 'guard') would tell the whole story. My guess would be that procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) would be enough. Cannot tell without specific data. > > For other threads, Rust doesn't try to remap the guard page, it just relies > on the guard page created by libthr in _thr_stack_alloc. > > Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's stack > overflow detection worked in 10.3. > > -Alan > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Mar 30 18:14:12 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B45155A81A1 for ; Tue, 30 Mar 2021 18:14:12 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8yJN3nlTz558Z for ; Tue, 30 Mar 2021 18:14:12 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: by mailman.nyi.freebsd.org (Postfix) id 821E25A81A0; Tue, 30 Mar 2021 18:14:12 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 81E485A819F for ; Tue, 30 Mar 2021 18:14:12 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8yJM5GpYz556g; Tue, 30 Mar 2021 18:14:10 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 12UIE2Ji088087 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 30 Mar 2021 11:14:02 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 12UIE25s088086; Tue, 30 Mar 2021 11:14:02 -0700 (PDT) (envelope-from jmg) Date: Tue, 30 Mar 2021 11:14:02 -0700 From: John-Mark Gurney To: Emanuel Haupt Cc: hackers@FreeBSD.org Subject: Re: RFC: possible issue with kqueue Message-ID: <20210330181402.GM14975@funkthat.com> Mail-Followup-To: Emanuel Haupt , hackers@FreeBSD.org References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Tue, 30 Mar 2021 11:14:02 -0700 (PDT) X-Rspamd-Queue-Id: 4F8yJM5GpYz556g X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-1.79 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[208.87.223.18:from]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[208.87.223.18:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.988]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 18:14:12 -0000 Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 +0100: > Can someone familiar with kqueue please comment on: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 Done: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 Looks like the user wasn't force unmounting the FS. There really isn't any problem w/ kqueue, as a normal unmount is expected to be refused while files are open. I guess there COULD be a new flag added to file descriptors that flag them as being able to be closed upon unmount. Then when an unmount happens and only these flagged files remain, they are closed allowing the fs to unmount. But this is a new feature and independent of kqueue. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@freebsd.org Tue Mar 30 18:51:12 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9FCDC5A9AB7 for ; Tue, 30 Mar 2021 18:51:12 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8z743mnZz588S for ; Tue, 30 Mar 2021 18:51:12 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 817995A9B39; Tue, 30 Mar 2021 18:51:12 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 814045A9DDC for ; Tue, 30 Mar 2021 18:51:12 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound5c.ore.mailhop.org (outbound5c.ore.mailhop.org [54.244.192.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8z741sCtz583x for ; Tue, 30 Mar 2021 18:51:11 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1617130270; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=rBTewYrdFtWJRYhrC6fF9XfNVegCKV/L7xqYxp7PxmU0Ps8tzwoJp7UR2Yjl5/RZgQtbeAupH7lh2 edgF2/kM705s9WbVG3UnBwJX+kufLzrUPUYmV+mgdCZicwtZO7LC8Hv1r04VAvl5mJTV1xskuzjFXl tlbtVfr0uEpspWrNz3p5C40NX5PcB4G46Yw4NzFzjrmj6Ae7Y9csNXAP2ARFozysDz6Ro9jf50kJzK 1JqEQQMo8XmrgTZD9dbqOZI9wakUhiiH393Ju5BPEdhoMwJV1c8vF3PdZHJGIfVMrDSgDqF2JugRGC B8qYZM6sUw2ZTA3TKBIqgOgOMmJ2Rjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=KS7l6obHgWlW/R+B0PZzwKaaT9YBgi2PcO82oYBrucM=; b=IChb68A/6aIcnsnbYzECdsm7tpOATlpnCUdsIP9bClI8/i8aiM1syn79qXlu7BDpEdnqIZ15AbI0B z0eQ4pX6yAdry8akZ9sYOfkVqze+2P57fnls0iyA9xhLQCGPPv5TNecrlMFLrRLSQ9fGsQCn+Z5TIy bn4JpFy0Y+6JQkmEbqboUg8fsOPphDt7fnnkLs/b5g4mRcl35XI0r1gNNSP+56H8rWZDbGnCH9Pks0 3MBAacHJ0hbDz3BqNc160hjpYu1/MYcOnah3/WPcOhr0AOgYzy5GH6gd0gHDPa7HHGWfv/QxMjbL2R bkkYBWcPp1XiDsB0zh9fdyzdtUvNZmw== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=KS7l6obHgWlW/R+B0PZzwKaaT9YBgi2PcO82oYBrucM=; b=R8LxFKHczvhYPZ227zHdY09wHe4iJ16zW9ICXdD51sQpD20rRESYNgUYj6XNGZqcOFYZ9ExKWAeHV L4+ibO85I6mi8uxDkIeXaQH/jyfp4iJCv/9hd/WtlmV6X0+wY5GUCz9p4Kn5hXHgarh/3YSm1Wlqwb NzNS4w97JYetjqGRHroIZpP4amh9c7WpoVFC63bAwkP3t/mohyFWBLFHOVrwlUCrDdDdDC42dRsZVg AC/rKwIzMRQPyjr+A954hEdlfo7A+NW8QnaZU62vqZEDp/FNZGWNfADWdTwIWSHHJfzkWKyMcpj22m S7K9T1zaCxYOQ/o5GE+jZ196B6N3ucg== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: e3567dd8-9188-11eb-aa8a-bf9d68d023b6 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id e3567dd8-9188-11eb-aa8a-bf9d68d023b6; Tue, 30 Mar 2021 18:51:09 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 12UIp7o7063547; Tue, 30 Mar 2021 12:51:07 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: RFC: possible issue with kqueue From: Ian Lepore To: John-Mark Gurney , Emanuel Haupt Cc: hackers@FreeBSD.org Date: Tue, 30 Mar 2021 12:51:07 -0600 In-Reply-To: <20210330181402.GM14975@funkthat.com> References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4F8z741sCtz583x X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 18:51:12 -0000 On Tue, 2021-03-30 at 11:14 -0700, John-Mark Gurney wrote: > Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 +0100: > > Can someone familiar with kqueue please comment on: > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 > > Done: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 > > Looks like the user wasn't force unmounting the FS. There really > isn't any problem w/ kqueue, as a normal unmount is expected to be > refused while files are open. > > I guess there COULD be a new flag added to file descriptors that > flag them as being able to be closed upon unmount. Then when an > unmount happens and only these flagged files remain, they are closed > allowing the fs to unmount. But this is a new feature and > independent > of kqueue. > While it's not a kqueue bug per se, it is a weakness in freebsd that there is no way to monitor a volume, or items within that volume, without making it impossible to unmount the volume. I've been fighting this with various implementations of desktop software for like 20 years on freebsd. Usually I have to just disable all monitoring and live with the reduced desktop functionality. I wonder if a reasonable fix might be to have some sort of pre-unmount event that can be delivered via kqueue, so that a userland entity monitoring on that volume has a chance to close related descriptors so that the unmount can proceed? -- Ian From owner-freebsd-hackers@freebsd.org Tue Mar 30 20:27:28 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4FB215AC9F3 for ; Tue, 30 Mar 2021 20:27:28 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F91G74KxJz3JBl for ; Tue, 30 Mar 2021 20:27:27 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wr1-x42d.google.com with SMTP id o16so17556244wrn.0 for ; Tue, 30 Mar 2021 13:27:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=48xM3R8D+ESgY/+zJEbGEeg9mlMfEFnbyCwemoIQn8U=; b=bit56zDJqeODF3k3PZGmuMh/jruD/ob0WcIEYvQq6scg8PjyVAmssqFa3xgWGS97HL z4yvbQyql4X0EQJQx6192EY0hfPPG/J/GJ0BfJWQnZ+G7F1CzXiZry8BcY+H52HihOI5 uiTaU/DsApN/qMY+eI0/LDdArXDUKI9kFNKWhW2ND/H3ol6FGV4OCnCQ3mkngNIAlHWH Dr5WeaEggvvr6WCdXHRJetbC1PHypJe9xq531yuQl9hzwvQY+KIqKVuVwb5vjdQSg/Sb jxHQ4MqzLZF3JEsTdCY1w2VPLqU7CgXadc1aJkGawhXaNBNEnJ5/0+1Yv/Dj4mIoxVhH KDrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=48xM3R8D+ESgY/+zJEbGEeg9mlMfEFnbyCwemoIQn8U=; b=XmYEMeK2hlJl3H4GxVYZjejGh6J/5RCYLDk0PGM5o52Dp4FqNi1MCJwLo5+Ra8YML6 hwM+XY+4/W0qEvYlJqCUssGhWXkUOYwwGzaBxdkuo0FwtCfNK73zCXaIjF89I5f+OyIp uMglwz3UQB2OShWNUV0KMLwRcSYrS/pXMgxu7i0jsnJPKyviEOusaBumI/NMjbG0fL+F ERl7aSOe1z1tHOF8eW0dCulIoAuwAg9ISN0DCTNKbzpTViNor2E6QZu3QvE71+oCxyX9 J8VmF8xNACaMxaIfhXak/CElrx7UiRcqKi6AGvndbNmPxjCrs12hKKCbG2OSvlsSTYCW +eSQ== X-Gm-Message-State: AOAM531pDyGrU8dd2t9Wr6PoXcdWs+mYQVjHgu5Y2wdNrO12KZA7A8jJ wiyaKte27uN5MT0vsjWmH/6gJsa4FJIu5w== X-Google-Smtp-Source: ABdhPJxUJyrIZ6AEkgjQs9pf3f29lVO8pmT+EumgRaYA/orC0K0ZEB4qMkGepvUHaC2ycvFZTJA8BA== X-Received: by 2002:adf:8151:: with SMTP id 75mr35582032wrm.152.1617136044115; Tue, 30 Mar 2021 13:27:24 -0700 (PDT) Received: from [192.168.1.13] (88-105-96-80.dynamic.dsl.as9105.com. [88.105.96.80]) by smtp.gmail.com with ESMTPSA id n7sm31495386wrv.71.2021.03.30.13.27.23 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 30 Mar 2021 13:27:23 -0700 (PDT) Subject: Re: RFC: possible issue with kqueue To: freebsd-hackers@freebsd.org References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> From: Graham Perrin Message-ID: <6e61f7fb-1f43-57c1-aa64-4ef7cb38d7bc@gmail.com> Date: Tue, 30 Mar 2021 21:27:22 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <20210330181402.GM14975@funkthat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-Rspamd-Queue-Id: 4F91G74KxJz3JBl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=bit56zDJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::42d as permitted sender) smtp.mailfrom=grahamperrin@gmail.com X-Spamd-Result: default: False [-2.01 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RECEIVED_SPAMHAUS_PBL(0.00)[88.105.96.80:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::42d:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::42d:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.99)[0.988]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::42d:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 20:27:28 -0000 On 30/03/2021 19:14, John-Mark Gurney wrote: > Looks like the user wasn't force unmounting the FS. I very rarely apply force without knowing what's open. Some technical skill was required to discover what was open. It's quicker and easier for me to kill the gvfsd-trash process. Not complaining; just saying :-) From owner-freebsd-hackers@freebsd.org Tue Mar 30 21:01:48 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 026125AD972 for ; Tue, 30 Mar 2021 21:01:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F921l4tBvz3LT3 for ; Tue, 30 Mar 2021 21:01:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id A756E5AD4C0; Tue, 30 Mar 2021 21:01:47 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A719B5AD743 for ; Tue, 30 Mar 2021 21:01:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F921l494Fz3LV8 for ; Tue, 30 Mar 2021 21:01:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82b.google.com with SMTP id f12so12998417qtq.4 for ; Tue, 30 Mar 2021 14:01:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Q4UDaRFj+wpfmyI63IoIplA0ujhIRlxKndjUipLxijE=; b=NKFfnlpKrZZU40vOLZTwtoZEMk5yHUGY8P4In5XaX7/5GRoCB3Cdvper2bnln5CTkC +yFGVd4cUar9tfL5EUJJjPN70eCI5zvzEyWKTom+H1jcS6NKm2f1ffb+sxeDh/x3F6UG 0eWDPCIPrkgf6SUoEYqAS+f7e63VtMkY4jthL9HNo/H10BEgs22Gcz8gbP4Vt3/S3fH1 4OhKGl1EMqzoSMoBpW8w2OwaWwsj7RWS6ECZ83j03eqpvfXLZqGKAnT8kej5inyn6kjP MmaXBKm5BnsOrgsrM1FSw+SnkfIS7go7+JnWxlgbHPPABAnQdbkq1R9uB0lCfK1hV/rn mafg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Q4UDaRFj+wpfmyI63IoIplA0ujhIRlxKndjUipLxijE=; b=e0YLcsxCu9dS+/Vc03DZTpAu+TkifizbFQ6hHlNSHJTQp/oPjacwGsuAsXgQuHjrCj Yhy3yGlPuaaqrHXKFYbDGZWoDYb1O3cJZBt+uGUDVaM91RxhaTlzv7gebtoRRTL5sp86 wMwPRo/sogMa3T4tW1Zttgf/mAUtXa4r+SI8O+ol+9o2sL56y71w2AYZ2E8BIbYP1AFO sD1sND/DyA9EC/0wbyCYXVvRm3jChr2Wl3ZHQO+q0T8+1Y/se9E2rWbX/Grf44JDeK6G U+XxBMWVG09HKPdJRqdcO6m3A7wixvV552ICZz1IJ61SaTK0Lp1HhBYHOSCziQFVSFwE uUKg== X-Gm-Message-State: AOAM5315XXu23+GZ/clQGdOUJ1JVBM/vR2kw6ffeHfZ7E3JWJR/Yl8Hu fsUzjuwPlozLz7O7Xa0J/QNF6++6SMLiIHEzBjndrg== X-Google-Smtp-Source: ABdhPJxqoatf9hEROtlgqF45I9V+7+IGSeFfrySCLUTUPCvMrjrQhWH7Rv4EyPJ4wFGgZx7p9Ep14YwUN9VuWsBrtvw= X-Received: by 2002:a05:622a:3c8:: with SMTP id k8mr29541325qtx.101.1617138106582; Tue, 30 Mar 2021 14:01:46 -0700 (PDT) MIME-Version: 1.0 References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> In-Reply-To: From: Warner Losh Date: Tue, 30 Mar 2021 15:01:35 -0600 Message-ID: Subject: Re: RFC: possible issue with kqueue To: Ian Lepore Cc: John-Mark Gurney , Emanuel Haupt , "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 4F921l494Fz3LV8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 21:01:48 -0000 On Tue, Mar 30, 2021 at 12:51 PM Ian Lepore wrote: > On Tue, 2021-03-30 at 11:14 -0700, John-Mark Gurney wrote: > > Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 +0100: > > > Can someone familiar with kqueue please comment on: > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 > > > > Done: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 > > > > Looks like the user wasn't force unmounting the FS. There really > > isn't any problem w/ kqueue, as a normal unmount is expected to be > > refused while files are open. > > > > I guess there COULD be a new flag added to file descriptors that > > flag them as being able to be closed upon unmount. Then when an > > unmount happens and only these flagged files remain, they are closed > > allowing the fs to unmount. But this is a new feature and > > independent > > of kqueue. > > > > While it's not a kqueue bug per se, it is a weakness in freebsd that > there is no way to monitor a volume, or items within that volume, > without making it impossible to unmount the volume. I've been fighting > this with various implementations of desktop software for like 20 years > on freebsd. Usually I have to just disable all monitoring and live > with the reduced desktop functionality. > > I wonder if a reasonable fix might be to have some sort of pre-unmount > event that can be delivered via kqueue, so that a userland entity > monitoring on that volume has a chance to close related descriptors so > that the unmount can proceed? > That's not a bad idea. I've often thought we need a wider range of quiesce calls / events / whatever that would allow people with 'soft' references to drop them in advance of an attempted event. I've never been sure what to do if the attempted event failed, so I've not gotten off the mark... this is but one example... Warner From owner-freebsd-hackers@freebsd.org Tue Mar 30 21:05:03 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD3595AD853 for ; Tue, 30 Mar 2021 21:05:03 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F925W59Nlz3Lf5 for ; Tue, 30 Mar 2021 21:05:03 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: by mailman.nyi.freebsd.org (Postfix) id B15D05AD491; Tue, 30 Mar 2021 21:05:03 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B127E5AD6E2 for ; Tue, 30 Mar 2021 21:05:03 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F925W37pzz3Lbw; Tue, 30 Mar 2021 21:05:02 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 12UL50Z3098868 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 30 Mar 2021 14:05:00 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 12UL50Zk098867; Tue, 30 Mar 2021 14:05:00 -0700 (PDT) (envelope-from jmg) Date: Tue, 30 Mar 2021 14:05:00 -0700 From: John-Mark Gurney To: Ian Lepore Cc: Emanuel Haupt , hackers@FreeBSD.org Subject: Re: RFC: possible issue with kqueue Message-ID: <20210330210500.GO14975@funkthat.com> Mail-Followup-To: Ian Lepore , Emanuel Haupt , hackers@FreeBSD.org References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Tue, 30 Mar 2021 14:05:00 -0700 (PDT) X-Rspamd-Queue-Id: 4F925W37pzz3Lbw X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 21:05:03 -0000 Ian Lepore wrote this message on Tue, Mar 30, 2021 at 12:51 -0600: > On Tue, 2021-03-30 at 11:14 -0700, John-Mark Gurney wrote: > > Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 +0100: > > > Can someone familiar with kqueue please comment on: > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 > > > > Done: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 > > > > Looks like the user wasn't force unmounting the FS. There really > > isn't any problem w/ kqueue, as a normal unmount is expected to be > > refused while files are open. > > > > I guess there COULD be a new flag added to file descriptors that > > flag them as being able to be closed upon unmount. Then when an > > unmount happens and only these flagged files remain, they are closed > > allowing the fs to unmount. But this is a new feature and > > independent > > of kqueue. > > > > While it's not a kqueue bug per se, it is a weakness in freebsd that > there is no way to monitor a volume, or items within that volume, > without making it impossible to unmount the volume. I've been fighting > this with various implementations of desktop software for like 20 years > on freebsd. Usually I have to just disable all monitoring and live > with the reduced desktop functionality. > > I wonder if a reasonable fix might be to have some sort of pre-unmount > event that can be delivered via kqueue, so that a userland entity > monitoring on that volume has a chance to close related descriptors so > that the unmount can proceed? Why not the soft close flag that I propose? It's nice and simple in that you just have to flag the various fd's in the app as opposed to add additional logic, and figure out how to tell the app to close the fd's... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@freebsd.org Tue Mar 30 21:05:09 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DDA625AD9BE for ; Tue, 30 Mar 2021 21:05:09 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F925d5BKVz3LfK for ; Tue, 30 Mar 2021 21:05:09 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id B1C225AD493; Tue, 30 Mar 2021 21:05:09 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B16105AD9BD for ; Tue, 30 Mar 2021 21:05:09 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound4s.ore.mailhop.org (outbound4s.ore.mailhop.org [54.185.97.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F925d2Dydz3LkG for ; Tue, 30 Mar 2021 21:05:08 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1617138307; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=NWe2OYxGexHFIHPwlnbEWL9XtJpthpBDMrk60Cw7UzoCA64Z1iRUfq0MpK2Vc9vzbpkHKstaPPsaG TJakkFpAnIvGA+lGeXMpWraphw+QZLly9z6INPeG/3gjFuP2/PdWv1UAA2KLbbA5hrJAqFT56Y/Aan av2uai3Kn8V3tua3/MnpRTiamDalpkVfEOeY/IZXYYggdy+XoIt3P6DkoAgmWQueaPlHFZN6mytxdG WzFa3QesU+kFMQqxfBsyVPYccFR3GEnSakG40iGH4dvFA2OcqPQRnQHyPUaInN/SZqK+ijIZyPAi9A hvY4XL5mcB+Iwlc8fOH9t+Cvmk7TTdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=OddVdHNksZO69Gc+ovbeqO90XQ81EEMpmtaUdB/C+Rk=; b=hF1etdwmXW/GisBoxXY1NnbNwZiG1h4oiJa0pwi5rVPORp92avMUZHi12K42b8TtX76BBQtJ7O8w/ Qu3sbMjmfPKlXiADnRmfSWS3VtTNk5qIV2k/rfaT5HtCOV7uyYxrNmbEZa5eKFQTI1VYaDxji0MEng wAPwpXmb/gHn0kCE6IjcNByRhDJBR/TjT8jgWW+yZi9IEt2/1DPvzEFAgIbvPEzDWtZmA9psyuEM/H vCbGW10dmtKG5RgBTSTxi7VlK/Z30TCCpPONxKq3bPP84+xK/Y8Kqe555ccEGpnniMLL9B6MyJptFb 8Vd4IBcfgyraazWdwP1kbK/nxTqEiuQ== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=OddVdHNksZO69Gc+ovbeqO90XQ81EEMpmtaUdB/C+Rk=; b=gZcAg6DxebZA+77J178uKdv6CdOldgZjBS8PCR7JOFvOdckTRSPYDRbqENw5RkHuoQ4dAYOpwmSTk QuV28PZWiCv451GEDSlilus0ALSVbK0iDbLjZFznifXLb0FD9nJ4s7/cQQC3SLcpasxrXngNIXpkud 2J3dmTW09O0+/YA8vDYX0V4mZYUQEfhZbJ7TH8IdJxYhdjxerjORjYNqIE2G6wsSy+lKL+1B08MQjr YZ+xSU07cWsXT3cfdVpK5o2qJkpbHDIWqxxta5EEEa0Jle6NIelyt88Dc64pngXGMAweuwMlWo6moY Dn8PkBQYAm701f3rrjJ8nh2U02P3EZw== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: 9988230f-919b-11eb-a600-89389772cfc7 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 9988230f-919b-11eb-a600-89389772cfc7; Tue, 30 Mar 2021 21:05:06 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 12UL54nJ064048; Tue, 30 Mar 2021 15:05:04 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: RFC: possible issue with kqueue From: Ian Lepore To: Warner Losh Cc: John-Mark Gurney , Emanuel Haupt , "freebsd-hackers@freebsd.org" Date: Tue, 30 Mar 2021 15:05:04 -0600 In-Reply-To: References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4F925d2Dydz3LkG X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 21:05:10 -0000 On Tue, 2021-03-30 at 15:01 -0600, Warner Losh wrote: > On Tue, Mar 30, 2021 at 12:51 PM Ian Lepore wrote: > > > On Tue, 2021-03-30 at 11:14 -0700, John-Mark Gurney wrote: > > > Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 > > > +0100: > > > > Can someone familiar with kqueue please comment on: > > > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 > > > > > > Done: > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 > > > > > > Looks like the user wasn't force unmounting the FS. There really > > > isn't any problem w/ kqueue, as a normal unmount is expected to > > > be > > > refused while files are open. > > > > > > I guess there COULD be a new flag added to file descriptors that > > > flag them as being able to be closed upon unmount. Then when an > > > unmount happens and only these flagged files remain, they are > > > closed > > > allowing the fs to unmount. But this is a new feature and > > > independent > > > of kqueue. > > > > > > > While it's not a kqueue bug per se, it is a weakness in freebsd > > that > > there is no way to monitor a volume, or items within that volume, > > without making it impossible to unmount the volume. I've been > > fighting > > this with various implementations of desktop software for like 20 > > years > > on freebsd. Usually I have to just disable all monitoring and live > > with the reduced desktop functionality. > > > > I wonder if a reasonable fix might be to have some sort of pre- > > unmount > > event that can be delivered via kqueue, so that a userland entity > > monitoring on that volume has a chance to close related descriptors > > so > > that the unmount can proceed? > > > > That's not a bad idea. I've often thought we need a wider range of > quiesce > calls / events / whatever that would allow people with 'soft' > references to > drop > them in advance of an attempted event. I've never been sure what to > do if > the attempted event failed, so I've not gotten off the mark... this > is but > one > example... > > Warner Yeah, I was kinda wondering what do about the case where the unmount ultimately failed. The only thought I had was some kind of followup event to inform folks of that (either a fake mount event, or a specific unmount-failed event). -- Ian From owner-freebsd-hackers@freebsd.org Wed Mar 31 02:28:23 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 989345B7E4C for ; Wed, 31 Mar 2021 02:28:23 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F99Gb2PM7z4RMk for ; Wed, 31 Mar 2021 02:28:23 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 506AE5B7E4B; Wed, 31 Mar 2021 02:28:23 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 502EB5B7E4A for ; Wed, 31 Mar 2021 02:28:23 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F99Gb1lYbz4RV9 for ; Wed, 31 Mar 2021 02:28:22 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-f169.google.com with SMTP id 184so22032172ljf.9 for ; Tue, 30 Mar 2021 19:28:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zRWxaK/SAqSBT2BLLzcaev/CvRCTEAeUw0JLC0P75VU=; b=F9QSVblvyr0TUJ/m2/BmqSEGROMduDnFkqMUeYYLKCDlonqyV0xik2zLOXoC/65fhv zLDz7yFIvPTDBUqkU451BpoFDuvtHguLTG1Dd67HM+RFh8CkpjM+11H16LDw/Gcj+AuZ kgybt4qcThYYKfFnJgiktVR6qv3MJcIEN+ZeOwkLJWT4wn80GdpEC8YZkKU/Hh748Leu Ia0qnG9X7iI6a5HOr+EPi9a2ZVNXp7uaUd4GtRUY/Jm/ifVfQQqxuCaEFmXxff9l6/5U URle1/54+pirnsrIgIPlT9UzrTv96e5LH6XJc+8+a6xbS16X1lYO+s7VgTADGp6PwyrS 944g== X-Gm-Message-State: AOAM533YBeAqPzgb8VjWqO+sBs6jJbgS1nacGWCSQ9f/8KJWXIHrEWWP MgmgzCq/5fhCSDbzXvqEsSApW3J7ecwXSpcjHMU= X-Google-Smtp-Source: ABdhPJyO+NA3km34qjai3DM/RO6SR7132eLyC8opRIjAJNhYklUovuqPheNMIOhH3qmHHx5mkrhZdzIGQJ16PXEUe0o= X-Received: by 2002:a2e:7619:: with SMTP id r25mr615020ljc.408.1617157700823; Tue, 30 Mar 2021 19:28:20 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Tue, 30 Mar 2021 20:28:09 -0600 Message-ID: Subject: Re: How does the stack's guard page work on amd64? To: Konstantin Belousov Cc: "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 4F99Gb1lYbz4RV9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2021 02:28:23 -0000 On Tue, Mar 30, 2021 at 3:35 AM Konstantin Belousov wrote: > On Mon, Mar 29, 2021 at 11:06:36PM -0600, Alan Somers wrote: > > Rust tries to detect stack overflow and handles it differently than other > > segfaults, but it's currently broken on FreeBSD/amd64. I've got a patch > > that fixes the problem, but I would like someone to confirm my reasoning. > > > > It seems like FreeBSD's main thread stacks include a guard page at the > > bottom. However, when Rust tries to create its own guard page (by > > re-mmap()ping and mprotect()ing it), it seems like FreeBSD's guard page > > automatically moves up into the un-remapped region. At least, that's how > > it behaves, based on the addresses that segfault. Is that correct? > Show the facts. For instance, procstat -v (and a note which > mapping was established by runtime for the 'guard') would tell the whole > story. > > My guess would be that procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) > would be enough. Cannot tell without specific data. > > > > > For other threads, Rust doesn't try to remap the guard page, it just > relies > > on the guard page created by libthr in _thr_stack_alloc. > > > > Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's stack > > overflow detection worked in 10.3. > > > > -Alan > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > Here is the relevant portion of procstat -v for a test program built with the buggy rustc: 651 0x801554000 0x80155d000 rw- 0 17 3 0 ----- df 651 0x801600000 0x801e00000 rw- 30 30 1 0 ----- df 651 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- -- 651 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- -- <--- What Rustc thinks is the guard page 651 0x7fffdffff000 0x7fffe0000000 --- 0 0 0 0 ----- -- <--- Where did this come from? 651 0x7fffe0000000 0x7fffe001e000 rw- 30 30 1 0 ---D- df 651 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- df Rustc tries to create that guard page by finding the base address of the stack, reallocating one page, then mprotect()ing it, like this: mmap(0x7fffdfffe000,0x1000,0x3,0x1012,0xffffffff,0) mprotect(0x7fffdfffe000,0x1000,0) If I patch rustc to not attempt to allocate a guard page, then its memory map looks like this. Notice that 0x7fffdffff000 is now accessible 662 0x801531000 0x80155b000 rw- 3 17 3 0 ----- df 662 0x801600000 0x801e00000 rw- 30 30 1 0 ----- df 662 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- -- 662 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- -- 662 0x7fffdffff000 0x7fffe001e000 rw- 31 31 1 0 ---D- df 662 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- df So the real question is, why does 0x7fffdffff000 become protected when rustc protects 0x7fffdfffe000 ? -Alan From owner-freebsd-hackers@freebsd.org Wed Mar 31 05:46:04 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 56A635BBEE2 for ; Wed, 31 Mar 2021 05:46:04 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F9Ffh0vHdz4cF5 for ; Wed, 31 Mar 2021 05:46:04 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 1CB165BBD43; Wed, 31 Mar 2021 05:46:04 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1C7305BBBC4 for ; Wed, 31 Mar 2021 05:46:04 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9Ffg2mtBz4cPv; Wed, 31 Mar 2021 05:46:02 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mail-pl1-x630.google.com with SMTP id f17so7334415plr.0; Tue, 30 Mar 2021 22:46:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eDX2QWhynwBUy2rLVoXu/o0Q6HVQmYTRbN/j6SwjO8w=; b=BLMrjAIrl24wjauYhhgIgeOlokQU+5+evmopMcl/l7pZ4yiIWUVA6ihGA7B52A2/fs Qygo6wU/TmAN5aK95soTHIsu+bfq/8rRtw86HhY1Oh86RsknRyMzBcoMNPbtmEZiBOmv fLEdat0agv6PjAcrCVmE4T3zPGgpfeyXrpBmyMsIzgw2Vjafct5IM1fBBxsEO3Q8/5J7 BlrFSLQZ0y84ISEHKFCkcw/HPSjjqQzPm9hAdG4FaPe5ooxvNqqPYcGY2JvQIH1NR5d+ px52k6ARvy7r7GYNWpQZFSsTCvsAmVzVmnqZdQnqJzNgxtFG1gNCL2DAUg5AzbX3DJ2P lyxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eDX2QWhynwBUy2rLVoXu/o0Q6HVQmYTRbN/j6SwjO8w=; b=DC73zqWZ79MAcTUuVPxRhSt4mL2As64lcrJAowTvSKx3lBRFj7jKe6mMH5C3GQOjbr 3KRdaN/stHjeYsBl2LPpeMjWfhDhlDDP8pHfcBYJPM3TLulXSG8SJB6EdFdJLQiAAvs0 0jXWZl0k19oZlanIPbkGal/bn2XBXh3SYzBKWWGGuMAU2v0+iR15fXru+6yhRy90UDdN hIp6Fy3OL7cXj+j8WjcWrMRlAIeJ5281ec4YqqST74HrrLzQamkgwiD/+hqGs4BZCbdH tCdDv4wZXWLNMVQSDsTFlJ4rl27S6FpYZCVkFa52p8mi/NJXSKdnIqHn5mXolheNBKUv qkPQ== X-Gm-Message-State: AOAM532Yv2JELKWzA7Wg3GXQQ2Fy1C6DUjqs9uOmcyBJrwp4hisjCYYl +6Q5OyeQ11JH53/gqZPpDb9UEAKPxVgWEr3qOtMXNM2FIDQ= X-Google-Smtp-Source: ABdhPJxDE4OQhtVNyklC79BUZNiCRfM8HUe/2U9C4dpU47mrWsSx9rqS0Kx95wj03q69tNtZbrIhlJXYx06u+idZUnw= X-Received: by 2002:a17:90a:c20a:: with SMTP id e10mr1747206pjt.221.1617169561326; Tue, 30 Mar 2021 22:46:01 -0700 (PDT) MIME-Version: 1.0 References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> In-Reply-To: <20210330181402.GM14975@funkthat.com> From: Damjan Jovanovic Date: Wed, 31 Mar 2021 07:45:49 +0200 Message-ID: Subject: Re: RFC: possible issue with kqueue To: Emanuel Haupt Cc: hackers@freebsd.org X-Rspamd-Queue-Id: 4F9Ffg2mtBz4cPv X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=BLMrjAIr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of damjanjov@gmail.com designates 2607:f8b0:4864:20::630 as permitted sender) smtp.mailfrom=damjanjov@gmail.com X-Spamd-Result: default: False [-3.98 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.98)[-0.984]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::630:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::630:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::630:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2021 05:46:04 -0000 On Tue, Mar 30, 2021 at 8:14 PM John-Mark Gurney wrote: > Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 +0100: > > Can someone familiar with kqueue please comment on: > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 > > Done: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 > > Looks like the user wasn't force unmounting the FS. There really > isn't any problem w/ kqueue, as a normal unmount is expected to be > refused while files are open. > > I guess there COULD be a new flag added to file descriptors that > flag them as being able to be closed upon unmount. Then when an > unmount happens and only these flagged files remain, they are closed > allowing the fs to unmount. But this is a new feature and independent > of kqueue. > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > > Linux's inotify avoids this problem by monitoring filesystem paths instead of file descriptors, which also has the advantage of not contributing to the open file limit. Can we do something like that? Damjan From owner-freebsd-hackers@freebsd.org Wed Mar 31 11:21:31 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 84BEF5C43D6 for ; Wed, 31 Mar 2021 11:21:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F9P5l1ybKz3CGl for ; Wed, 31 Mar 2021 11:21:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 433575C43D5; Wed, 31 Mar 2021 11:21:31 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42FBB5C43D4 for ; Wed, 31 Mar 2021 11:21:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9P5k6x8lz3C3M; Wed, 31 Mar 2021 11:21:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 12VBLGKc077447 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 31 Mar 2021 14:21:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 12VBLGKc077447 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 12VBLGlZ077446; Wed, 31 Mar 2021 14:21:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 31 Mar 2021 14:21:16 +0300 From: Konstantin Belousov To: Alan Somers Cc: "freebsd-hackers@freebsd.org" Subject: Re: How does the stack's guard page work on amd64? Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4F9P5k6x8lz3C3M X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2021 11:21:31 -0000 On Tue, Mar 30, 2021 at 08:28:09PM -0600, Alan Somers wrote: > On Tue, Mar 30, 2021 at 3:35 AM Konstantin Belousov > wrote: > > > On Mon, Mar 29, 2021 at 11:06:36PM -0600, Alan Somers wrote: > > > Rust tries to detect stack overflow and handles it differently than other > > > segfaults, but it's currently broken on FreeBSD/amd64. I've got a patch > > > that fixes the problem, but I would like someone to confirm my reasoning. > > > > > > It seems like FreeBSD's main thread stacks include a guard page at the > > > bottom. However, when Rust tries to create its own guard page (by > > > re-mmap()ping and mprotect()ing it), it seems like FreeBSD's guard page > > > automatically moves up into the un-remapped region. At least, that's how > > > it behaves, based on the addresses that segfault. Is that correct? > > Show the facts. For instance, procstat -v (and a note which > > mapping was established by runtime for the 'guard') would tell the whole > > story. > > > > My guess would be that procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) > > would be enough. Cannot tell without specific data. > > > > > > > > For other threads, Rust doesn't try to remap the guard page, it just > > relies > > > on the guard page created by libthr in _thr_stack_alloc. > > > > > > Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's stack > > > overflow detection worked in 10.3. > > > > > > -Alan > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to " > > freebsd-hackers-unsubscribe@freebsd.org" > > > > Here is the relevant portion of procstat -v for a test program built with > the buggy rustc: > 651 0x801554000 0x80155d000 rw- 0 17 3 0 ----- df > 651 0x801600000 0x801e00000 rw- 30 30 1 0 ----- df > 651 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- -- > 651 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- -- > <--- What Rustc thinks is the guard page > 651 0x7fffdffff000 0x7fffe0000000 --- 0 0 0 0 ----- -- > <--- Where did this come from? This is the stack grow area, occupied by 'elastic' guard entry. It serves two purposes: 1. it keeps the space, preventing other non-fixed mappings from selecting the grow area for mapping. 2. it prevents stack from growing down to the next mapping below it, preventing issues like StackClash. See mmap(2) esp. MAP_STACK part of it. > 651 0x7fffe0000000 0x7fffe001e000 rw- 30 30 1 0 ---D- df > 651 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- df > > Rustc tries to create that guard page by finding the base address of the > stack, reallocating one page, then mprotect()ing it, like this: > mmap(0x7fffdfffe000,0x1000,0x3,0x1012,0xffffffff,0) > mprotect(0x7fffdfffe000,0x1000,0) > > If I patch rustc to not attempt to allocate a guard page, then its memory > map looks like this. Notice that 0x7fffdffff000 is now accessible It is accessible because stack grown down into this address. > 662 0x801531000 0x80155b000 rw- 3 17 3 0 ----- df > 662 0x801600000 0x801e00000 rw- 30 30 1 0 ----- df > 662 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- -- > 662 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- -- > 662 0x7fffdffff000 0x7fffe001e000 rw- 31 31 1 0 ---D- df > 662 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- df > > So the real question is, why does 0x7fffdffff000 become protected when > rustc protects 0x7fffdfffe000 ? See above. As I said in earlier response, if you want fully shrinkable stack guard, set procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) during runtime initialization. Or better, do not create custom guard page at all, relying on system guard. From owner-freebsd-hackers@freebsd.org Wed Mar 31 13:11:34 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F0F805C79DF for ; Wed, 31 Mar 2021 13:11:34 +0000 (UTC) (envelope-from Administrator@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F9RXk6cFDz3Kmc for ; Wed, 31 Mar 2021 13:11:34 +0000 (UTC) (envelope-from Administrator@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id E2D2F5C7967; Wed, 31 Mar 2021 13:11:34 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E29F45C7AA5 for ; Wed, 31 Mar 2021 13:11:34 +0000 (UTC) (envelope-from Administrator@freebsd.org) Received: from mta0.pwc-1lc.com (hwsrv-849161.hostwindsdns.com [192.236.154.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9RXk5Gwrz3Kks for ; Wed, 31 Mar 2021 13:11:34 +0000 (UTC) (envelope-from Administrator@freebsd.org) From: Administrator To: hackers@freebsd.org Subject: Pending Email Delivering Date: 31 Mar 2021 13:11:29 +0000 Message-ID: <20210331131129.554C859AD44DD93F@freebsd.org> X-Rspamd-Queue-Id: 4F9RXk5Gwrz3Kks X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:54290, ipnet:192.236.154.0/23, country:US]; local_wl_from(0.00)[freebsd.org] MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2021 13:11:35 -0000 From owner-freebsd-hackers@freebsd.org Thu Apr 1 04:06:42 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D1B685BB2AD for ; Thu, 1 Apr 2021 04:06:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F9qPZ3tMHz3KK7 for ; Thu, 1 Apr 2021 04:06:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 8518F5BB342; Thu, 1 Apr 2021 04:06:42 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 84DE95BB515 for ; Thu, 1 Apr 2021 04:06:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9qPZ3JT6z3KM6 for ; Thu, 1 Apr 2021 04:06:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f176.google.com with SMTP id n140so499915oig.9 for ; Wed, 31 Mar 2021 21:06:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gK2xXEgiaD9YN9uGKRXbL5c9u9mid9V0go19xba7Eog=; b=d/OaGm9GI28upiPMadYshB3NXI7zeAIeLhW6QdA3/3nmCGdT4l5/B+T8qLbzbBYBTq mCqgd4A/3PVIhePkCdRcspSmJ7M83gQ2pUGYDlI81F7gvPoMcy1N4ibmVculwXU0mc11 ox5woRZ5SWv4L0MrkuSjAwPR/WswUrZD03uh9boLwjTVb8Kx9OQqanO+HO7IN+m0nAuh Axo42qv0EvLMXXWS7WuNWvOpB7vAXXn4YPIE9lRaUvUq5Bz6trIdFcnjurkHoxwAQzd+ lqfoDs1yfTcbO6kAgDRvUW1HuhI7sXRM6qzy9448Yw7JPdSFfaQENQNRcM0bf7ZXqSt+ B2LA== X-Gm-Message-State: AOAM531f9Vu60tNXEycSkxgPIO+8zX9FglXFJD6uIT3Ebmkx3pSG1ujz aTg/aM8aSgsErcQr1vDFrHXnY1QF6iS1Uxcxbyo= X-Google-Smtp-Source: ABdhPJx8SPkJEiz31i2fBqrJtsDCYT2Mz2kWXpa3KGkOWQUTOsqMSgZMMcntl9kE/dePlTgf2LP2DgGc7hQ+HmEOB34= X-Received: by 2002:aca:b06:: with SMTP id 6mr4564043oil.73.1617250001141; Wed, 31 Mar 2021 21:06:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 31 Mar 2021 22:06:30 -0600 Message-ID: Subject: Re: How does the stack's guard page work on amd64? To: Konstantin Belousov Cc: "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 4F9qPZ3JT6z3KM6 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 04:06:42 -0000 On Wed, Mar 31, 2021 at 5:21 AM Konstantin Belousov wrote: > On Tue, Mar 30, 2021 at 08:28:09PM -0600, Alan Somers wrote: > > On Tue, Mar 30, 2021 at 3:35 AM Konstantin Belousov > > > wrote: > > > > > On Mon, Mar 29, 2021 at 11:06:36PM -0600, Alan Somers wrote: > > > > Rust tries to detect stack overflow and handles it differently than > other > > > > segfaults, but it's currently broken on FreeBSD/amd64. I've got a > patch > > > > that fixes the problem, but I would like someone to confirm my > reasoning. > > > > > > > > It seems like FreeBSD's main thread stacks include a guard page at > the > > > > bottom. However, when Rust tries to create its own guard page (by > > > > re-mmap()ping and mprotect()ing it), it seems like FreeBSD's guard > page > > > > automatically moves up into the un-remapped region. At least, > that's how > > > > it behaves, based on the addresses that segfault. Is that correct? > > > Show the facts. For instance, procstat -v (and a note which > > > mapping was established by runtime for the 'guard') would tell the > whole > > > story. > > > > > > My guess would be that procctl(PROC_STACKGAP_CTL, > &PROC_STACKGAP_DISABLE) > > > would be enough. Cannot tell without specific data. > > > > > > > > > > > For other threads, Rust doesn't try to remap the guard page, it just > > > relies > > > > on the guard page created by libthr in _thr_stack_alloc. > > > > > > > > Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's stack > > > > overflow detection worked in 10.3. > > > > > > > > -Alan > > > > _______________________________________________ > > > > freebsd-hackers@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > > To unsubscribe, send any mail to " > > > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > Here is the relevant portion of procstat -v for a test program built with > > the buggy rustc: > > 651 0x801554000 0x80155d000 rw- 0 17 3 0 ----- > df > > 651 0x801600000 0x801e00000 rw- 30 30 1 0 ----- > df > > 651 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- > -- > > 651 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- > -- > > <--- What Rustc thinks is the guard page > > 651 0x7fffdffff000 0x7fffe0000000 --- 0 0 0 0 ----- > -- > > <--- Where did this come from? > This is the stack grow area, occupied by 'elastic' guard entry. > It serves two purposes: > 1. it keeps the space, preventing other non-fixed mappings from selecting > the grow area for mapping. > 2. it prevents stack from growing down to the next mapping below it, > preventing issues like StackClash. > > See mmap(2) esp. MAP_STACK part of it. > I saw that. And I even saw where libthr uses MAP_STACK when creating new threads. However, this program is single-threaded. Where does the stack get created for a process's main thread? I couldn't find that. > > > 651 0x7fffe0000000 0x7fffe001e000 rw- 30 30 1 0 ---D- > df > > 651 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- > df > > > > Rustc tries to create that guard page by finding the base address of the > > stack, reallocating one page, then mprotect()ing it, like this: > > > mmap(0x7fffdfffe000,0x1000,0x3,0x1012,0xffffffff,0) > > mprotect(0x7fffdfffe000,0x1000,0) > > > > If I patch rustc to not attempt to allocate a guard page, then its memory > > map looks like this. Notice that 0x7fffdffff000 is now accessible > It is accessible because stack grown down into this address. > > > 662 0x801531000 0x80155b000 rw- 3 17 3 0 ----- > df > > 662 0x801600000 0x801e00000 rw- 30 30 1 0 ----- > df > > 662 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- > -- > > 662 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- > -- > > 662 0x7fffdffff000 0x7fffe001e000 rw- 31 31 1 0 ---D- > df > > 662 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- > df > > > > So the real question is, why does 0x7fffdffff000 become protected when > > rustc protects 0x7fffdfffe000 ? > See above. > > As I said in earlier response, if you want fully shrinkable stack guard, > set procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) during runtime > initialization. > > Or better, do not create custom guard page at all, relying on system guard. > That's what my patch does. But I've only tested it on amd64, and I don't have access to alternative architectures. Does every architecture create a stack guard this way? -Alan From owner-freebsd-hackers@freebsd.org Thu Apr 1 06:59:50 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 881145C8F65 for ; Thu, 1 Apr 2021 06:59:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F9vFL1rJmz4hdR for ; Thu, 1 Apr 2021 06:59:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 3D4505C8C6F; Thu, 1 Apr 2021 06:59:50 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D0245C8EF7 for ; Thu, 1 Apr 2021 06:59:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9vFK5z8Wz4hQJ; Thu, 1 Apr 2021 06:59:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1316xdgG058243 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 1 Apr 2021 09:59:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1316xdgG058243 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1316xdvI058242; Thu, 1 Apr 2021 09:59:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 1 Apr 2021 09:59:39 +0300 From: Konstantin Belousov To: Alan Somers Cc: "freebsd-hackers@freebsd.org" Subject: Re: How does the stack's guard page work on amd64? Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4F9vFK5z8Wz4hQJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 06:59:50 -0000 On Wed, Mar 31, 2021 at 10:06:30PM -0600, Alan Somers wrote: > On Wed, Mar 31, 2021 at 5:21 AM Konstantin Belousov > wrote: > > > On Tue, Mar 30, 2021 at 08:28:09PM -0600, Alan Somers wrote: > > > On Tue, Mar 30, 2021 at 3:35 AM Konstantin Belousov > > > > > wrote: > > > > > > > On Mon, Mar 29, 2021 at 11:06:36PM -0600, Alan Somers wrote: > > > > > Rust tries to detect stack overflow and handles it differently than > > other > > > > > segfaults, but it's currently broken on FreeBSD/amd64. I've got a > > patch > > > > > that fixes the problem, but I would like someone to confirm my > > reasoning. > > > > > > > > > > It seems like FreeBSD's main thread stacks include a guard page at > > the > > > > > bottom. However, when Rust tries to create its own guard page (by > > > > > re-mmap()ping and mprotect()ing it), it seems like FreeBSD's guard > > page > > > > > automatically moves up into the un-remapped region. At least, > > that's how > > > > > it behaves, based on the addresses that segfault. Is that correct? > > > > Show the facts. For instance, procstat -v (and a note which > > > > mapping was established by runtime for the 'guard') would tell the > > whole > > > > story. > > > > > > > > My guess would be that procctl(PROC_STACKGAP_CTL, > > &PROC_STACKGAP_DISABLE) > > > > would be enough. Cannot tell without specific data. > > > > > > > > > > > > > > For other threads, Rust doesn't try to remap the guard page, it just > > > > relies > > > > > on the guard page created by libthr in _thr_stack_alloc. > > > > > > > > > > Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's stack > > > > > overflow detection worked in 10.3. > > > > > > > > > > -Alan > > > > > _______________________________________________ > > > > > freebsd-hackers@freebsd.org mailing list > > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > > > To unsubscribe, send any mail to " > > > > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > > > > Here is the relevant portion of procstat -v for a test program built with > > > the buggy rustc: > > > 651 0x801554000 0x80155d000 rw- 0 17 3 0 ----- > > df > > > 651 0x801600000 0x801e00000 rw- 30 30 1 0 ----- > > df > > > 651 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- > > -- > > > 651 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- > > -- > > > <--- What Rustc thinks is the guard page > > > 651 0x7fffdffff000 0x7fffe0000000 --- 0 0 0 0 ----- > > -- > > > <--- Where did this come from? > > This is the stack grow area, occupied by 'elastic' guard entry. > > It serves two purposes: > > 1. it keeps the space, preventing other non-fixed mappings from selecting > > the grow area for mapping. > > 2. it prevents stack from growing down to the next mapping below it, > > preventing issues like StackClash. > > > > See mmap(2) esp. MAP_STACK part of it. > > > > I saw that. And I even saw where libthr uses MAP_STACK when creating new > threads. However, this program is single-threaded. Where does the stack > get created for a process's main thread? I couldn't find that. In kernel during execve(2). Specifically, sys/kern/kern_exec.c, exec_new_vmspace(), vm_map_stack() call. > > > > > > > 651 0x7fffe0000000 0x7fffe001e000 rw- 30 30 1 0 ---D- > > df > > > 651 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- > > df > > > > > > Rustc tries to create that guard page by finding the base address of the > > > stack, reallocating one page, then mprotect()ing it, like this: > > > > > mmap(0x7fffdfffe000,0x1000,0x3,0x1012,0xffffffff,0) > > > mprotect(0x7fffdfffe000,0x1000,0) > > > > > > If I patch rustc to not attempt to allocate a guard page, then its memory > > > map looks like this. Notice that 0x7fffdffff000 is now accessible > > It is accessible because stack grown down into this address. > > > > > 662 0x801531000 0x80155b000 rw- 3 17 3 0 ----- > > df > > > 662 0x801600000 0x801e00000 rw- 30 30 1 0 ----- > > df > > > 662 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 ----- > > -- > > > 662 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 ----- > > -- > > > 662 0x7fffdffff000 0x7fffe001e000 rw- 31 31 1 0 ---D- > > df > > > 662 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 ---D- > > df > > > > > > So the real question is, why does 0x7fffdffff000 become protected when > > > rustc protects 0x7fffdfffe000 ? > > See above. > > > > As I said in earlier response, if you want fully shrinkable stack guard, > > set procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) during runtime > > initialization. > > > > Or better, do not create custom guard page at all, relying on system guard. > > > > That's what my patch does. But I've only tested it on amd64, and I don't > have access to alternative architectures. Does every architecture create a > stack guard this way? Yes. From owner-freebsd-hackers@freebsd.org Fri Apr 2 05:00:11 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 02CE85C1571 for ; Fri, 2 Apr 2021 05:00:11 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FBSXp5104z3p8y for ; Fri, 2 Apr 2021 05:00:10 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id A9C715C1570; Fri, 2 Apr 2021 05:00:10 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A98B75C156F for ; Fri, 2 Apr 2021 05:00:10 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FBSXp4Ltlz3p6T for ; Fri, 2 Apr 2021 05:00:10 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f48.google.com with SMTP id h6-20020a0568300346b02901b71a850ab4so4085609ote.6 for ; Thu, 01 Apr 2021 22:00:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OyQIRg+s0fH/Ct5V135S2WmOJeMP/o7Pc4eREznJ/Ww=; b=f1QRc4z3aOdHevuDmDtHjV2jnz5QI1/AaCDeLeF1GzUimpH6s9b5wGJtDqqXte/KqO 686ni5PZzUm4N/qz2fvEioCBN/AD2tPAs6b1sqn07+7Y7GOZaAZyRC8LGlc7VXObWyvo h2EcTnw99ps8HBvpm7tA9HBjg4QoHj2uEytJNhFXUMgdS9AfrsgrXPMtETMUX+98gcr9 KIgpWMO6X2HrU6yUV1YxUtEG4aZp0S+NxNm+oK5HgjDL8IP7fOvthW3sRamW5NRbe+27 REdx6l9U6YlBhGG2SAbGSowYBPXPtu6N8NxhKZ6nqfnXDOt8MQyRp82IivkeB5isyqPn bXpg== X-Gm-Message-State: AOAM530vmB32D4pQfcajc5Zxa7n6J3lhL31RHe2QzLwdBDehwTiE8l22 OIp7S7I02EKTWf1U4JL1JPlxFK0lDjLA+01MiIui6VmQjdQ= X-Google-Smtp-Source: ABdhPJyIuF1quHPQg2h95E5hlC21oCfaHPKFJA5OCcIAmXyxs3PvN28MZHWVsGS1H2DMXChAb5UU6ut7LJGHVO8yeqw= X-Received: by 2002:a05:6830:1af6:: with SMTP id c22mr9500783otd.291.1617339609091; Thu, 01 Apr 2021 22:00:09 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 1 Apr 2021 22:59:57 -0600 Message-ID: Subject: Re: How does the stack's guard page work on amd64? To: Konstantin Belousov Cc: "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 4FBSXp4Ltlz3p6T X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2021 05:00:11 -0000 On Thu, Apr 1, 2021 at 12:59 AM Konstantin Belousov wrote: > On Wed, Mar 31, 2021 at 10:06:30PM -0600, Alan Somers wrote: > > On Wed, Mar 31, 2021 at 5:21 AM Konstantin Belousov > > > wrote: > > > > > On Tue, Mar 30, 2021 at 08:28:09PM -0600, Alan Somers wrote: > > > > On Tue, Mar 30, 2021 at 3:35 AM Konstantin Belousov < > kostikbel@gmail.com > > > > > > > > wrote: > > > > > > > > > On Mon, Mar 29, 2021 at 11:06:36PM -0600, Alan Somers wrote: > > > > > > Rust tries to detect stack overflow and handles it differently > than > > > other > > > > > > segfaults, but it's currently broken on FreeBSD/amd64. I've got > a > > > patch > > > > > > that fixes the problem, but I would like someone to confirm my > > > reasoning. > > > > > > > > > > > > It seems like FreeBSD's main thread stacks include a guard page > at > > > the > > > > > > bottom. However, when Rust tries to create its own guard page > (by > > > > > > re-mmap()ping and mprotect()ing it), it seems like FreeBSD's > guard > > > page > > > > > > automatically moves up into the un-remapped region. At least, > > > that's how > > > > > > it behaves, based on the addresses that segfault. Is that > correct? > > > > > Show the facts. For instance, procstat -v (and a note which > > > > > mapping was established by runtime for the 'guard') would tell the > > > whole > > > > > story. > > > > > > > > > > My guess would be that procctl(PROC_STACKGAP_CTL, > > > &PROC_STACKGAP_DISABLE) > > > > > would be enough. Cannot tell without specific data. > > > > > > > > > > > > > > > > > For other threads, Rust doesn't try to remap the guard page, it > just > > > > > relies > > > > > > on the guard page created by libthr in _thr_stack_alloc. > > > > > > > > > > > > Finally, what changed in between FreeBSD 10.3 and 11.4? Rust's > stack > > > > > > overflow detection worked in 10.3. > > > > > > > > > > > > -Alan > > > > > > _______________________________________________ > > > > > > freebsd-hackers@freebsd.org mailing list > > > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > > > > To unsubscribe, send any mail to " > > > > > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > > > > > > > Here is the relevant portion of procstat -v for a test program built > with > > > > the buggy rustc: > > > > 651 0x801554000 0x80155d000 rw- 0 17 3 0 > ----- > > > df > > > > 651 0x801600000 0x801e00000 rw- 30 30 1 0 > ----- > > > df > > > > 651 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 > ----- > > > -- > > > > 651 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 > ----- > > > -- > > > > <--- What Rustc thinks is the guard page > > > > 651 0x7fffdffff000 0x7fffe0000000 --- 0 0 0 0 > ----- > > > -- > > > > <--- Where did this come from? > > > This is the stack grow area, occupied by 'elastic' guard entry. > > > It serves two purposes: > > > 1. it keeps the space, preventing other non-fixed mappings from > selecting > > > the grow area for mapping. > > > 2. it prevents stack from growing down to the next mapping below it, > > > preventing issues like StackClash. > > > > > > See mmap(2) esp. MAP_STACK part of it. > > > > > > > I saw that. And I even saw where libthr uses MAP_STACK when creating new > > threads. However, this program is single-threaded. Where does the stack > > get created for a process's main thread? I couldn't find that. > In kernel during execve(2). Specifically, sys/kern/kern_exec.c, > exec_new_vmspace(), vm_map_stack() call. > > > > > > > > > > > > 651 0x7fffe0000000 0x7fffe001e000 rw- 30 30 1 0 > ---D- > > > df > > > > 651 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 > ---D- > > > df > > > > > > > > Rustc tries to create that guard page by finding the base address of > the > > > > stack, reallocating one page, then mprotect()ing it, like this: > > > > > > > > mmap(0x7fffdfffe000,0x1000,0x3,0x1012,0xffffffff,0) > > > > mprotect(0x7fffdfffe000,0x1000,0) > > > > > > > > If I patch rustc to not attempt to allocate a guard page, then its > memory > > > > map looks like this. Notice that 0x7fffdffff000 is now accessible > > > It is accessible because stack grown down into this address. > > > > > > > 662 0x801531000 0x80155b000 rw- 3 17 3 0 > ----- > > > df > > > > 662 0x801600000 0x801e00000 rw- 30 30 1 0 > ----- > > > df > > > > 662 0x7fffdfffd000 0x7fffdfffe000 --- 0 0 0 0 > ----- > > > -- > > > > 662 0x7fffdfffe000 0x7fffdffff000 --- 0 0 0 0 > ----- > > > -- > > > > 662 0x7fffdffff000 0x7fffe001e000 rw- 31 31 1 0 > ---D- > > > df > > > > 662 0x7fffe001e000 0x7fffe003e000 rw- 32 32 1 0 > ---D- > > > df > > > > > > > > So the real question is, why does 0x7fffdffff000 become protected > when > > > > rustc protects 0x7fffdfffe000 ? > > > See above. > > > > > > As I said in earlier response, if you want fully shrinkable stack > guard, > > > set procctl(PROC_STACKGAP_CTL, &PROC_STACKGAP_DISABLE) during runtime > > > initialization. > > > > > > Or better, do not create custom guard page at all, relying on system > guard. > > > > > > > That's what my patch does. But I've only tested it on amd64, and I don't > > have access to alternative architectures. Does every architecture > create a > > stack guard this way? > > Yes. > Thanks for the explanation. That led me to what has changed since 10.3: r320317 . I've opened a PR with rustc to fix the bug. Thanks for all your help. -Alan From owner-freebsd-hackers@freebsd.org Thu Apr 1 11:28:23 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E10CA5C988B for ; Thu, 1 Apr 2021 11:28:23 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FB1CC4jCGz3PQM for ; Thu, 1 Apr 2021 11:28:23 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: by mailman.nyi.freebsd.org (Postfix) id A160B5C9543; Thu, 1 Apr 2021 11:28:23 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A125D5C9542 for ; Thu, 1 Apr 2021 11:28:23 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: from mxt.nsu.ru (mxt.nsu.ru [84.237.50.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FB1C9522Bz3PWK for ; Thu, 1 Apr 2021 11:28:20 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: from mail.nsu.ru ([84.237.50.42] helo=zimbra.nsu.ru) by mxt.nsu.ru with esmtp (Exim 4.89) (envelope-from ) id 1lRvUl-00078w-QR for hackers@freebsd.org; Thu, 01 Apr 2021 18:28:16 +0700 Received: from localhost (localhost [127.0.0.1]) by zimbra.nsu.ru (Postfix) with ESMTP id C56D6AC0312 for ; Thu, 1 Apr 2021 18:28:15 +0700 (+07) Received: from zimbra.nsu.ru ([127.0.0.1]) by localhost (zimbra.nsu.ru [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 7Gc5RNfUWfYw for ; Thu, 1 Apr 2021 18:28:15 +0700 (+07) Received: from localhost (localhost [127.0.0.1]) by zimbra.nsu.ru (Postfix) with ESMTP id 8C113AC0558 for ; Thu, 1 Apr 2021 18:28:15 +0700 (+07) X-Virus-Scanned: amavisd-new at zimbra.nsu.ru Received: from zimbra.nsu.ru ([127.0.0.1]) by localhost (zimbra.nsu.ru [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id EDSbtyKW56iH for ; Thu, 1 Apr 2021 18:28:15 +0700 (+07) Received: from regency.nsu.ru (unknown [84.237.50.47]) by zimbra.nsu.ru (Postfix) with ESMTPS id 5FF2EAC0312 for ; Thu, 1 Apr 2021 18:28:15 +0700 (+07) Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.14.2/8.14.2) with ESMTP id 131BSGZb027048 for ; Thu, 1 Apr 2021 18:28:16 +0700 (+07) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.14.2/8.14.2/Submit) id 131BSAVb027001 for hackers@freebsd.org; Thu, 1 Apr 2021 18:28:11 +0700 (+07) (envelope-from danfe) Date: Thu, 1 Apr 2021 18:28:10 +0700 From: Alexey Dokuchaev To: hackers@freebsd.org Subject: Re: wcwidth() and wcswidth() and Latin vs. CJK character width Message-ID: <20210401112810.GA53432@regency.nsu.ru> References: <20210324143117.GA61738@regency.nsu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210324143117.GA61738@regency.nsu.ru> User-Agent: Mutt/1.4.2.1i X-KLMS-Rule-ID: 1 X-KLMS-Message-Action: clean X-KLMS-AntiSpam-Lua-Profiles: 162829 [Apr 01 2021] X-KLMS-AntiSpam-Version: 5.9.20.0 X-KLMS-AntiSpam-Envelope-From: danfe@regency.nsu.ru X-KLMS-AntiSpam-Rate: 0 X-KLMS-AntiSpam-Status: not_detected X-KLMS-AntiSpam-Method: none X-KLMS-AntiSpam-Auth: dmarc=pass header.from=nsu.ru policy=quarantine; spf=pass smtp.mailfrom=regency.nsu.ru; dkim=none X-KLMS-AntiSpam-Info: LuaCore: 441 441 d8b9a5b7b92fd8a4a285bba155aec05d4c327708, {rep_avail}, {Tracking_from_domain_doesnt_match_to}, zimbra.nsu.ru:7.1.1; 127.0.0.199:7.1.2; regency.nsu.ru:7.1.1; nsu.ru:7.1.1; 84.237.50.42:7.1.2; d41d8cd98f00b204e9800998ecf8427e.com:7.1.1, {Tracking_smtp_domain_mismatch}, ApMailHostAddress: 84.237.50.42 X-MS-Exchange-Organization-SCL: -1 X-KLMS-AntiSpam-Interceptor-Info: scan successful X-KLMS-AntiPhishing: Clean, bases: 2021/04/01 09:29:00 X-KLMS-AntiVirus: Kaspersky Security for Linux Mail Server, version 8.0.3.30, not scanned, license restriction X-Rspamd-Queue-Id: 4FB1C9522Bz3PWK X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.20 / 15.00]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:mxt.nsu.ru]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[nsu.ru:+]; DMARC_POLICY_ALLOW(-0.50)[nsu.ru,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[danfe@nsu.ru,danfe@regency.nsu.ru]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[84.237.50.40:from]; ASN(0.00)[asn:3335, ipnet:84.237.48.0/21, country:RU]; MIME_TRACE(0.00)[0:+]; FROM_NEQ_ENVFROM(0.00)[danfe@nsu.ru,danfe@regency.nsu.ru]; DWL_DNSWL_NONE(0.00)[nsu.ru:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[nsu.ru:s=email]; FREEFALL_USER(0.00)[danfe]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[84.237.50.40:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[84.237.50.40:from]; RCVD_COUNT_SEVEN(0.00)[8]; MAILMAN_DEST(0.00)[hackers] X-Mailman-Approved-At: Fri, 02 Apr 2021 08:01:00 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 11:28:23 -0000 On Wed, Mar 24, 2021 at 09:31:17PM +0700, Alexey Dokuchaev wrote: > Hi there, > > I've been wondering if there's anything like Markus Kuhn's implementation* > in our base libraries for $subj (my quick naive search didn't turn up any > results). If yes, could someone point me at it? If no, how feasible would > it be to add Markus' code to our libc or perhaps some ancillary library? So I guess lack of replies mean "please go ahead". :-) Before I open a DR for adding wcwidth.c to libutil, how do we handle single 3rd-party files, last modified in 2007? :) Just add them as is, apply any necessary changes, and commit (together with the manpage), or they must go through the vendor area like other contributed software? ./danfe From owner-freebsd-hackers@freebsd.org Fri Apr 2 20:52:14 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC5985B1F19 for ; Fri, 2 Apr 2021 20:52:14 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FBsgL3H7qz3txn for ; Fri, 2 Apr 2021 20:52:14 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 707465B1F18; Fri, 2 Apr 2021 20:52:14 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 703335B1F16 for ; Fri, 2 Apr 2021 20:52:14 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FBsgK58VWz3tlm for ; Fri, 2 Apr 2021 20:52:13 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f181.google.com with SMTP id d2so5499572ilm.10 for ; Fri, 02 Apr 2021 13:52:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=njzTGI5Hqa8y1z+M8wPg2lkMyNbG3HbRq0LcpxXt4Ho=; b=tmIEPB7A9pH1DmKGgtf5zL18UTezfY8I+RbxUu8Z91LFVV2ZImMPJz6izo3itYez+n 7UDDUbUJrkRiNpLDjDfa/rQ6nmve2OBTBYkM655FYJZm2/hz5iHvz1apPUNxLOVL8HJX exeiG1V5j1pvQxqlwwvSk8o1TkhdaWujPJ5LS3H9WKhVEr4FTGTHZqq9iROe66ezGZBM 78pz9R2ru0rvno9OfHSfEQsqx4xb7g3jWaxdVfoG8ab72g79paozsMSjJxmeyCqRQgk0 5Zb4feuq0WI4q2WXHWHw1mwu4Tlytm9CnpIa59se0QoJF1vUWMmFu+3GZ7CA48WQBvDt fBqg== X-Gm-Message-State: AOAM533HvtLfGx/tIPUU+gwk8keYVf/o+GIlU84nvPXrPFadH4wQrXap KrPrTjRAtUbfG67wRQawIsCA1PlrEdblq84tQBA= X-Google-Smtp-Source: ABdhPJwNx9rS/bIBf7FEsQjDvPZL7j3B3OVADGMFE4dYgKIkh6Up7lno6Nn0OUDfdubz0Ae+HTy9lT/mw6wmQEfDTXY= X-Received: by 2002:a05:6e02:85:: with SMTP id l5mr13004465ilm.182.1617396732200; Fri, 02 Apr 2021 13:52:12 -0700 (PDT) MIME-Version: 1.0 References: <20210324143117.GA61738@regency.nsu.ru> In-Reply-To: <20210324143117.GA61738@regency.nsu.ru> From: Ed Maste Date: Fri, 2 Apr 2021 16:51:49 -0400 Message-ID: Subject: Re: wcwidth() and wcswidth() and Latin vs. CJK character width To: Alexey Dokuchaev Cc: hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FBsgK58VWz3tlm X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.181 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_SHORT(-1.00)[-0.999]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.166.181:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; SPAMHAUS_ZRD(0.00)[209.85.166.181:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.181:from]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.181:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2021 20:52:14 -0000 On Wed, 24 Mar 2021 at 10:37, Alexey Dokuchaev via freebsd-hackers wrote: > > Hi there, > > I've been wondering if there's anything like Markus Kuhn's implementation* > in our base libraries for $subj (my quick naive search didn't turn up any > results). If yes, could someone point me at it? Kuhn's implementation is in the kernel for teken (sys/teken/teken_wcwidth.h). But, wcwidth/wcswidth are available already in userland, is there some reason you are looking at Kuhn's implementation specifically? From owner-freebsd-hackers@freebsd.org Sat Apr 3 03:00:04 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D2FCC5BC66B for ; Sat, 3 Apr 2021 03:00:04 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC1qm46hfz4mqW for ; Sat, 3 Apr 2021 03:00:04 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 8AA0E5BCB60; Sat, 3 Apr 2021 03:00:04 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8A4715BCB5F for ; Sat, 3 Apr 2021 03:00:04 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com [IPv6:2a00:1450:4864:20::232]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FC1ql2Ngyz4n0L; Sat, 3 Apr 2021 03:00:03 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-lj1-x232.google.com with SMTP id r20so7247618ljk.4; Fri, 02 Apr 2021 20:00:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Kz4nZc4dxx5VMxRao16MaE0vVgeBaoQxO5+5U2PfLCQ=; b=f3bEGQw3wcr6IADa03iAsoFKNAZgkNdoP84pLyb+q2eWJA6y0qk1sFlT78NUa5XdEm w+amgB2zcsOY/DyTHlq1Dizodj1U8G4QGqnn6YagYI99DsteQrjbtXFjZ26Xtk6EzR5e gL16lzBvXu+BzoB3JDE3f0yOQ6rN81+dQ9BovigVYRmbd/s+nEXxE20IjiUL0rFfiGRL SLPpUc0+X/d6uUZDBIGBnJ53hpbmd/t+R8SnxMlmQUDOzyMTg6cjGNEGYCyBBxg0KHip kxVljG/xYYxkNQX4E30wyiKCtoLVcgK9H/UMXjSLq8eKz7NqI8Con5qbPBwMeyGr6l24 rd/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Kz4nZc4dxx5VMxRao16MaE0vVgeBaoQxO5+5U2PfLCQ=; b=Y+p9gkGKz92GbNRm33SP6OtA8OGJU9pCnqZbq+LjbfTKOyYbIAScC/NQgdsonrlv9Y cRAZ/8ZmN0G7VUEhW7MxW1h0Ccj5264v1bzDxhT6iaJcx3mNAq2mrNB49g8hzj0Sg/Q5 8LanGyqnxSlvbmL4C0ltW6RKvPZKBD/BLvWNSnlkjvETfcUs2WKjNFGNaUTvCpeB0XcE 1xaUl2Il4oZVrXPxtkPPfdiTJvn7ReARUEM+eli0exI/pwh8v+wmywtzuIjzg7GBMNAi bMyIoSWiCTtAcIMBlI8VdModUoaHHatrnUy3xcO6lxiySJNuSxqPulvd4LkZxHLkguRo xE6Q== X-Gm-Message-State: AOAM530LR7sl392O0pOeYAHtf8JdJIYWB79J2D1UWWXYw/B2THOT6EW8 Doj4R3xLbgvQ90BDjhQngaGkbKRQL9g= X-Google-Smtp-Source: ABdhPJwJb9Lk2SfTihAIh4vrXUUoBqETQpKNex5OnrtLhODOqSmGWTKN6g5B+70o2OS9hdFA7kusfg== X-Received: by 2002:a2e:8987:: with SMTP id c7mr9867911lji.185.1617418801168; Fri, 02 Apr 2021 20:00:01 -0700 (PDT) Received: from rimwks.local ([2001:470:1f15:3d8:7285:c2ff:fe37:5722]) by smtp.gmail.com with ESMTPSA id i22sm1092087ljg.37.2021.04.02.19.59.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Apr 2021 20:00:00 -0700 (PDT) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Sat, 3 Apr 2021 05:59:54 +0300 To: Emanuel Haupt Cc: hackers@FreeBSD.org Subject: Re: RFC: possible issue with kqueue Message-ID: <20210403055954.5456c7f0@rimwks.local> In-Reply-To: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FC1ql2Ngyz4n0L X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=f3bEGQw3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rozhukim@gmail.com designates 2a00:1450:4864:20::232 as permitted sender) smtp.mailfrom=rozhukim@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::232:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::232:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::232:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 03:00:04 -0000 On Sat, 27 Mar 2021 13:10:11 +0100 Emanuel Haupt wrote: > Can someone familiar with kqueue please comment on: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 Because FreeBSD does not have O_NOATIME and O_EVTONLY, like MacOS. From owner-freebsd-hackers@freebsd.org Sat Apr 3 03:22:41 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5BE7C5BDC00 for ; Sat, 3 Apr 2021 03:22:41 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC2Ks0bZvz4pQK for ; Sat, 3 Apr 2021 03:22:41 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 1249F5BD5FF; Sat, 3 Apr 2021 03:22:41 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 120175BD5FE for ; Sat, 3 Apr 2021 03:22:41 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com [IPv6:2a00:1450:4864:20::232]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FC2Kr20y0z4pTL; Sat, 3 Apr 2021 03:22:40 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-lj1-x232.google.com with SMTP id u9so7238545ljd.11; Fri, 02 Apr 2021 20:22:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n04lM4p1CjXDeMEdt2ajEE86xxlqE+rNEUwEqx5kPog=; b=BwJY5eGH25HELJ2Z+JYNfYbkBrvlFrRVuJbvZzXpRIBoaNWQw+3Ujied8F5bU9B/cN tjuR1+0a1CLloYCrFhEF0miHGjUdF/a52VpBB0+mC4y4DrPf8PfPvvi5GyuVdN9xbLAo YZdHfwg5aKesjG1Gt+t/7gJcR2tbdFj44EzbryuJhU6BDVZXcNtgYN38wEmBoo/tSUlX LnkIyADg5FJQckdKy/pbryi/UR53emH+i5BqTEOBEnpIj7y5MMY0P/MNXbVRX3CbCtQn pKga3CJfwxgLKr5rqKPMhP9u/UPa3Q5DROQrrqsxE90DR5UbNuLmlWIn961eGB2DaVZ0 AoYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n04lM4p1CjXDeMEdt2ajEE86xxlqE+rNEUwEqx5kPog=; b=LS4NOVm4tflCnhjdqektPihQoCvlsdW0l3T8hr0V6Jj0EcX7BG6RTxRTel9FvenT5w YtUZ4FsFSeZCtFszndgXI9v4djhTdAainKQSC/gqqnPUc+J0FgFY5u7vQjwQA/3VLhiX gY5PEEVlxN8H2Cp+1nkLOfCtnz4uiBTyH9WOOTxkeXDH3bCjMcOFWmyJK2y+tATC54rO fmtkKRCdMP/8CKTaTzOmFtr/eeFQXW8Ceq+/b0gfsH5Ofd4fBKFL9Y9wUP9nrlPcnbtL biuMRQRadov1ipgOhjJaweh7gyMDHaZHH4RPumtXmimADchqcWIEAJzvcvC9v5rEEc7I 3Tmg== X-Gm-Message-State: AOAM533tE5ybl22lqKonCYBZ+fDbBFv3I46iTVIjDtwMm40/9PTtiBjr Uuk9RY+nuEJTaQBGDI2E3tJLXC5Id3EOLQ== X-Google-Smtp-Source: ABdhPJxMH+rr+ZV4W98ceTB9VoCBbS1bczEC+J4nfdROj2QA/KSZlTOUb3sUGkMC6dFMduJJT+BaTA== X-Received: by 2002:a2e:8006:: with SMTP id j6mr9506977ljg.417.1617420159011; Fri, 02 Apr 2021 20:22:39 -0700 (PDT) Received: from rimwks.local ([2001:470:1f15:3d8:7285:c2ff:fe37:5722]) by smtp.gmail.com with ESMTPSA id g6sm1028487lfh.232.2021.04.02.20.22.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Apr 2021 20:22:38 -0700 (PDT) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Sat, 3 Apr 2021 06:22:36 +0300 To: Ian Lepore Cc: John-Mark Gurney , Emanuel Haupt , hackers@FreeBSD.org Subject: Re: RFC: possible issue with kqueue Message-ID: <20210403062236.526fa1c4@rimwks.local> In-Reply-To: References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FC2Kr20y0z4pTL X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=BwJY5eGH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rozhukim@gmail.com designates 2a00:1450:4864:20::232 as permitted sender) smtp.mailfrom=rozhukim@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::232:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::232:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::232:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 03:22:41 -0000 > I wonder if a reasonable fix might be to have some sort of pre-unmount > event that can be delivered via kqueue, so that a userland entity > monitoring on that volume has a chance to close related descriptors so > that the unmount can proceed? > https://reviews.freebsd.org/D19690 I do not finish this. Plan was: implement this and catch unmount notify in my FAM, that in glib20. From owner-freebsd-hackers@freebsd.org Sat Apr 3 07:43:46 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1C3545C2A2D for ; Sat, 3 Apr 2021 07:43:46 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC87567VHz3LnJ for ; Sat, 3 Apr 2021 07:43:45 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id D02C35C2D8E; Sat, 3 Apr 2021 07:43:45 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE8A85C2F8E for ; Sat, 3 Apr 2021 07:43:45 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FC8750LLMz3LwG for ; Sat, 3 Apr 2021 07:43:44 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: by mail-lf1-f46.google.com with SMTP id d12so10332238lfv.11 for ; Sat, 03 Apr 2021 00:43:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FS1ATyTkrPrGYs3u6eCMBNhNwmjlr9J87eMedVzIaL0=; b=kIEMXp+nAwdmF9ezL9vQoEbA3hYEYcbbiz9CHcwJpSLzrcNZFkm9JFpjvlX+KU0Hbg dCii5yRPGPdkBEx+yDSGl/vpP4ycVlrxZgLRlvIkOgzscB+pe/gySP2mgnAZB0KAq9Tk clj8Nnt85TiVB/BRmEk5bJjSq+OWeVP/8U4N7an/wOkJeJtE3xAHhwEGvIZBdtRluSHv qqRH+34H/Us2abv7KFelI7OPaJ91swbJ+/DULPEqRHGB+fnNK1JsaReo/zFjb1stTIZa YgO7SFQPNTLwSsN6Q6CE/0h7spASGbD4kpeZgM8C0nT42zzrdSxl03dkIwG/pvYQqyzw 5WdA== X-Gm-Message-State: AOAM530trxZC4DgjFJ4goKee5mKHts+pLPVNa1Bap4IBktvJSjzAEfPY 7L06lM+dsSCvf8zbPRLaxsHjnmStKtZQbg== X-Google-Smtp-Source: ABdhPJzueiPYEzOTniF7jpQQfvdCp4OTysJYqbP90yqN8XxtSqTQW+U98vTEezDspd2P7c0dxKMJkA== X-Received: by 2002:ac2:41c4:: with SMTP id d4mr10733899lfi.334.1617435822646; Sat, 03 Apr 2021 00:43:42 -0700 (PDT) Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com. [209.85.208.176]) by smtp.gmail.com with ESMTPSA id b34sm1146841ljf.137.2021.04.03.00.43.42 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 03 Apr 2021 00:43:42 -0700 (PDT) Received: by mail-lj1-f176.google.com with SMTP id u9so7597222ljd.11 for ; Sat, 03 Apr 2021 00:43:42 -0700 (PDT) X-Received: by 2002:a2e:8881:: with SMTP id k1mr10214610lji.441.1617435822188; Sat, 03 Apr 2021 00:43:42 -0700 (PDT) MIME-Version: 1.0 References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> In-Reply-To: From: Gleb Popov Date: Sat, 3 Apr 2021 10:43:16 +0300 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: RFC: possible issue with kqueue To: Damjan Jovanovic , Rozhuk Ivan Cc: hackers@freebsd.org X-Rspamd-Queue-Id: 4FC8750LLMz3LwG X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of 6yearold@gmail.com designates 209.85.167.46 as permitted sender) smtp.mailfrom=6yearold@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[arrowd@freebsd.org,6yearold@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.167.46:from]; R_DKIM_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[arrowd@freebsd.org,6yearold@gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; SPAMHAUS_ZRD(0.00)[209.85.167.46:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.46:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.46:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 07:43:46 -0000 On Wed, Mar 31, 2021 at 8:46 AM Damjan Jovanovic wrote: > On Tue, Mar 30, 2021 at 8:14 PM John-Mark Gurney wrote: > > > Emanuel Haupt wrote this message on Sat, Mar 27, 2021 at 13:10 +0100: > > > Can someone familiar with kqueue please comment on: > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024 > > > > Done: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254024#c11 > > > > Looks like the user wasn't force unmounting the FS. There really > > isn't any problem w/ kqueue, as a normal unmount is expected to be > > refused while files are open. > > > > I guess there COULD be a new flag added to file descriptors that > > flag them as being able to be closed upon unmount. Then when an > > unmount happens and only these flagged files remain, they are closed > > allowing the fs to unmount. But this is a new feature and independent > > of kqueue. > > > > -- > > John-Mark Gurney Voice: +1 415 225 5579 > > > > > > > Linux's inotify avoids this problem by monitoring filesystem paths instead > of file descriptors, which also has the advantage of not contributing to > the open file limit. Can we do something like that? > The "O_PATH" flag support is being cooked in https://reviews.freebsd.org/D29323 , maybe it'd be possible to build something upon it. However, I was under the impression that the Linux advantage is the ability to set a single watch on a whole directory, while kqueue requires opening each file. On Sat, Apr 3, 2021 at 6:22 AM Rozhuk Ivan wrote: > > https://reviews.freebsd.org/D19690 > > I do not finish this. > Plan was: implement this and catch unmount notify in my FAM, that in > glib20. > Hum, but I do see mount/unmount events in devd on CURRENT! From owner-freebsd-hackers@freebsd.org Sat Apr 3 06:34:28 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AA0DB5C17D4 for ; Sat, 3 Apr 2021 06:34:28 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC6b83CPxz3HJH for ; Sat, 3 Apr 2021 06:34:28 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: by mailman.nyi.freebsd.org (Postfix) id 6BC5B5C17D3; Sat, 3 Apr 2021 06:34:28 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A3625C1647 for ; Sat, 3 Apr 2021 06:34:28 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: from mxt.nsu.ru (mxt.nsu.ru [84.237.50.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FC6b80mXgz3HJG; Sat, 3 Apr 2021 06:34:27 +0000 (UTC) (envelope-from danfe@regency.nsu.ru) Received: from zimbra.nsu.ru ([84.237.50.42]) by mxt.nsu.ru with esmtp (Exim 4.89) (envelope-from ) id 1lSZrT-0006zC-3t; Sat, 03 Apr 2021 13:34:23 +0700 Received: from localhost (localhost [127.0.0.1]) by zimbra.nsu.ru (Postfix) with ESMTP id 192D0AC0240; Sat, 3 Apr 2021 13:34:23 +0700 (+07) Received: from zimbra.nsu.ru ([127.0.0.1]) by localhost (zimbra.nsu.ru [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id dn61fqI4rCUF; Sat, 3 Apr 2021 13:34:22 +0700 (+07) Received: from localhost (localhost [127.0.0.1]) by zimbra.nsu.ru (Postfix) with ESMTP id 936CCAC18C6; Sat, 3 Apr 2021 13:34:22 +0700 (+07) X-Virus-Scanned: amavisd-new at zimbra.nsu.ru Received: from zimbra.nsu.ru ([127.0.0.1]) by localhost (zimbra.nsu.ru [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Gv8Vr1HV2Osj; Sat, 3 Apr 2021 13:34:22 +0700 (+07) Received: from regency.nsu.ru (unknown [84.237.50.47]) by zimbra.nsu.ru (Postfix) with ESMTPS id 5DC1DAC0240; Sat, 3 Apr 2021 13:34:22 +0700 (+07) Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.14.2/8.14.2) with ESMTP id 1336YSPN058210; Sat, 3 Apr 2021 13:34:28 +0700 (+07) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.14.2/8.14.2/Submit) id 1336YNnE058142; Sat, 3 Apr 2021 13:34:23 +0700 (+07) (envelope-from danfe) Date: Sat, 3 Apr 2021 13:34:23 +0700 From: Alexey Dokuchaev To: Ed Maste Cc: hackers@freebsd.org Subject: Re: wcwidth() and wcswidth() and Latin vs. CJK character width Message-ID: <20210403063423.GA50619@regency.nsu.ru> References: <20210324143117.GA61738@regency.nsu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i X-KLMS-Rule-ID: 1 X-KLMS-Message-Action: clean X-KLMS-AntiSpam-Lua-Profiles: 162882 [Apr 02 2021] X-KLMS-AntiSpam-Version: 5.9.20.0 X-KLMS-AntiSpam-Envelope-From: danfe@regency.nsu.ru X-KLMS-AntiSpam-Rate: 0 X-KLMS-AntiSpam-Status: not_detected X-KLMS-AntiSpam-Method: none X-KLMS-AntiSpam-Auth: dmarc=pass header.from=nsu.ru policy=quarantine; spf=pass smtp.mailfrom=regency.nsu.ru; dkim=none X-KLMS-AntiSpam-Info: LuaCore: 442 442 b985cb57763b61d2a20abb585d5d4cc10c315b09, {rep_avail}, {Tracking_from_domain_doesnt_match_to}, nsu.ru:7.1.1; d41d8cd98f00b204e9800998ecf8427e.com:7.1.1; zimbra.nsu.ru:7.1.1; 127.0.0.199:7.1.2; 84.237.50.42:7.1.2; regency.nsu.ru:7.1.1, {Tracking_smtp_domain_mismatch}, ApMailHostAddress: 84.237.50.42 X-MS-Exchange-Organization-SCL: -1 X-KLMS-AntiSpam-Interceptor-Info: scan successful X-KLMS-AntiPhishing: Clean, bases: 2021/04/03 05:49:00 X-KLMS-AntiVirus: Kaspersky Security for Linux Mail Server, version 8.0.3.30, not scanned, license restriction X-Rspamd-Queue-Id: 4FC6b80mXgz3HJG X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Mailman-Approved-At: Sat, 03 Apr 2021 07:58:12 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 06:34:28 -0000 On Fri, Apr 02, 2021 at 04:51:49PM -0400, Ed Maste wrote: > On Wed, 24 Mar 2021 at 10:37, Alexey Dokuchaev via freebsd-hackers > wrote: > > I've been wondering if there's anything like Markus Kuhn's > > implementation* in our base libraries for $subj (my quick naive > > search didn't turn up any results). If yes, could someone point > > me at it? > > Kuhn's implementation is in the kernel for teken > (sys/teken/teken_wcwidth.h). But, wcwidth/wcswidth are available > already in userland, Our default wcwidth(3) does not seem to work as described, but then I've found that Kuhn's code is also part of the libxo(3) which what I should have probably used in the first place. > is there some reason you are looking at Kuhn's implementation > specifically? I want to improve the "ifconfig wlan0 list scan" output, e.g. it currently does not display SSIDs which are in Russian or CJK, but conversion to libxo(3) would address both this problem and make the output easily consumable by external tools. ./danfe From owner-freebsd-hackers@freebsd.org Sat Apr 3 10:22:55 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5F4405C72FF for ; Sat, 3 Apr 2021 10:22:55 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FCCfl14l4z3mcr for ; Sat, 3 Apr 2021 10:22:55 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: by mailman.nyi.freebsd.org (Postfix) id 244CB5C72FE; Sat, 3 Apr 2021 10:22:55 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 240585C78E6 for ; Sat, 3 Apr 2021 10:22:55 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FCCfl03xmz3mwP; Sat, 3 Apr 2021 10:22:54 +0000 (UTC) (envelope-from yuripv@yuripv.dev) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 62B485C009A; Sat, 3 Apr 2021 06:22:54 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Sat, 03 Apr 2021 06:22:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.dev; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=F wJbzdSy/ZY1uarHMUcVnwvCOi7Uv+xjXz8Y3oloRnE=; b=lLUZ5f55BvZnOhEE+ FCmg9vCv6y2VEqwVYvxVi5J6bpbPuWB5tWG+tCnBsY83yEjTtH6xpuQR2nFq+po1 hhzApY/1njEz9n4SCoNldWNg/nqnmCa2vx1OcnwQ1jUjKin2xigBOj3Vi8zEYCrj x6JMuyi4t/IoERTCfi8X8GYvWXc/XhHg8sZcFrtQcAgWnyDuq3BD05aqa0f1xrW9 Zlk9e9kR4ZrxDmDrPucK3gi273yXneEnuivBiVqRyhPUwCw7CS/zQqRvnvGagrH1 FfNIVKgp10b0PGZ9f4/JZ1XIA6gtxghNslrjSB5v9X9Yh5EEe2GzlALhXNegdQZ3 OC4Rw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=FwJbzdSy/ZY1uarHMUcVnwvCOi7Uv+xjXz8Y3oloR nE=; b=coFUwrL4maXqmspt0mlvyJ0FDc+qj+fVg7LH94V3PMf6wEnLu2NzqWj5r ohufy2QPazI8NLov+PdQ/IxnTzqMcePVLMtPHxRUzIGiMsA+INkrFXePZNJYgSH2 l5Z9ad/nsZ0qQhiF5gIYnzZ1GiHnIvR0/ThN+yxWpQ+BbEuf/onnDWg4i39JnTcf pRYDm89JBpSB2H1xAyLCCvY7j7NDMoJh1x7ON1kxWlQ6MjNPUrfr2meHyFiP7xdk a4mby0IO8gqyn/xhrEuBTLFT5+gzP7XH4QZ5oMz8xz6f5WBkHCCpqsr4qPmMUw/h P3khnsoTLYAgWa0/kFgjX/5YegLmw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudeikedgfeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhkffffgggjggtgfesthejredttdefjeenucfhrhhomhepjghurhhi ucfrrghnkhhovhcuoeihuhhrihhpvheshihurhhiphhvrdguvghvqeenucggtffrrghtth gvrhhnpeffhedvkeegieejveelvdffgfevueevgffgjefhleeuueevtedvvddvgfeiteff teenucfkphepledurddvgedtrdduvdegrddufeejnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhephihurhhiphhvseihuhhrihhpvhdruggvvh X-ME-Proxy: Received: from [192.168.1.12] (unknown [91.240.124.137]) by mail.messagingengine.com (Postfix) with ESMTPA id 3DA80240054; Sat, 3 Apr 2021 06:22:53 -0400 (EDT) Subject: Re: wcwidth() and wcswidth() and Latin vs. CJK character width To: Alexey Dokuchaev , Ed Maste Cc: hackers@freebsd.org References: <20210324143117.GA61738@regency.nsu.ru> <20210403063423.GA50619@regency.nsu.ru> From: Yuri Pankov Message-ID: Date: Sat, 3 Apr 2021 13:22:50 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <20210403063423.GA50619@regency.nsu.ru> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FCCfl03xmz3mwP X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 10:22:55 -0000 Alexey Dokuchaev via freebsd-hackers wrote: > On Fri, Apr 02, 2021 at 04:51:49PM -0400, Ed Maste wrote: >> On Wed, 24 Mar 2021 at 10:37, Alexey Dokuchaev via freebsd-hackers >> wrote: >>> I've been wondering if there's anything like Markus Kuhn's >>> implementation* in our base libraries for $subj (my quick naive >>> search didn't turn up any results). If yes, could someone point >>> me at it? >> >> Kuhn's implementation is in the kernel for teken >> (sys/teken/teken_wcwidth.h). But, wcwidth/wcswidth are available >> already in userland, > > Our default wcwidth(3) does not seem to work as described, but then > I've found that Kuhn's code is also part of the libxo(3) which what > I should have probably used in the first place. > >> is there some reason you are looking at Kuhn's implementation >> specifically? > > I want to improve the "ifconfig wlan0 list scan" output, e.g. it > currently does not display SSIDs which are in Russian or CJK, but > conversion to libxo(3) would address both this problem and make > the output easily consumable by external tools. I think this could be easily fixed by adding setlocale(LC_ALL, "") call to ifconfig's main(), wcwidth() won't work properly without correct locale set (at least, without LC_CTYPE). From owner-freebsd-hackers@freebsd.org Sat Apr 3 17:22:02 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E21475B9ECD for ; Sat, 3 Apr 2021 17:22:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FCNyL4FcKz4htq for ; Sat, 3 Apr 2021 17:22:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id 9193F5B9DFA; Sat, 3 Apr 2021 17:22:02 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9133A5B9DF8 for ; Sat, 3 Apr 2021 17:22:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FCNyL3R04z4hxC for ; Sat, 3 Apr 2021 17:22:01 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x836.google.com with SMTP id l13so5696007qtu.9 for ; Sat, 03 Apr 2021 10:22:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JuoXT+BUtdwDhjW9tyNJBN7c+FExljhelrdYAxhwN6c=; b=EpHOlz5bOZVPdgq/eZguza67JmSP+IaRWSXt1CyEGDV0bXDbqNJZ3ySYHBmcQQ7SmO rFccWJjC/x6pOW14EGT4ZXEZ+3dznQVUEuLWonsEaxol/5iJLKzzdpQxMptPkbz1n1iA 71UX1g+V37ytvOSybTeq/corKvj1XKKFfjOPKf5tVTi2m46cvxdQTOkgjVR1e8h8HMpu Fb1HaoJRfTWaiEQpUSp2E3TenbXFPd6L7N5/m4Q8p/ttOtSIEp4EVlyMWAX3k5519JKY L0jSdp9rigKhMtAu+YodwgvA5HZqW4Wswsrlif3JwMjzcGhl4fDlCFvHhOxupTfCMqbL I8jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JuoXT+BUtdwDhjW9tyNJBN7c+FExljhelrdYAxhwN6c=; b=KA+0dzM+f85DsPKuojSQBQ8bxO8VK9MOAz5PRk6kCaWi/YO3FTX/yVF8OzP1CV4MkE TMWQT4Nz4GrLd4IV6meIMKIzsDItPSHyNsKhtO+FgGzfLtJXB/LpGSt1jNlPc5K9k+2J 5KARNVGWgeKSSgwdj+iDMdyCPmqYcV+6JuIH8k4HRF7CR5xWvQH6OQ/rBbmimvkKiNso qFIQn/c1YMvSXgMyTsOtudW+t+oQS0pdIJ/sF0Pgc5f3q/BLAUICzra+R5TTwLP3R9Uz UF5Mdf7CFmTikBFUJsumwt+5ZNE38ketQHAJphjWjAFROkmUSOajZENjH/Cx292EvQUQ k70Q== X-Gm-Message-State: AOAM5316NOdQipW3mk/yS4Dpa9P4ppqjPKfSdLdFjjN0384KJkhYU/3M Ipi60v3Xk69sfkZNZUbhW0MKJjGHHOEzn7hDYbBQxQ== X-Google-Smtp-Source: ABdhPJyuCdPCWJl31DcMxkYiThpvVjYjnKE8oXVg3VnKX5EGWOrBKqlHPXQEYerLqfRCBBTF6RBLolMnwLU6CdQGOvc= X-Received: by 2002:a05:622a:1748:: with SMTP id l8mr16125733qtk.73.1617470521306; Sat, 03 Apr 2021 10:22:01 -0700 (PDT) MIME-Version: 1.0 References: <20210327131011.e16291cac86475e75a33812c@FreeBSD.org> <20210330181402.GM14975@funkthat.com> In-Reply-To: From: Warner Losh Date: Sat, 3 Apr 2021 11:21:50 -0600 Message-ID: Subject: Re: RFC: possible issue with kqueue To: Gleb Popov Cc: Damjan Jovanovic , Rozhuk Ivan , "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: 4FCNyL3R04z4hxC X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 17:22:03 -0000 On Sat, Apr 3, 2021 at 1:44 AM Gleb Popov wrote: > On Sat, Apr 3, 2021 at 6:22 AM Rozhuk Ivan wrote: > > > > > https://reviews.freebsd.org/D19690 > > > > I do not finish this. > > Plan was: implement this and catch unmount notify in my FAM, that in > > glib20. > > > > Hum, but I do see mount/unmount events in devd on CURRENT! > I did this in conjunction with the forced unmount work of chs@. I was unaware of this review, but this commit, and those that followed it, added a similar feature. Warner commit 8ef773d1b4236ed3da368f9c91d15c5325d2e735 Author: Warner Losh Date: Wed Aug 19 17:10:04 2020 +0000 Add VFS FS events for mount and unmount to devctl/devd Report when a filesystem is mounted, remounted or unmounted via devd, along with details about the mount point and mount options. Discussed with: kib@ Reviewed by: kirk@ (prior version) Sponsored by: Netflix Diffential Revision: https://reviews.freebsd.org/D25969 Notes: svn path=/head/; revision=364402