From owner-freebsd-threads@freebsd.org Sun Jun 14 21:00:22 2020 Return-Path: Delivered-To: freebsd-threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 249C0344E2E for ; Sun, 14 Jun 2020 21:00:22 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 49lRgT6k9Xz4sg8 for ; Sun, 14 Jun 2020 21:00:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id E6A1A344B57; Sun, 14 Jun 2020 21:00:21 +0000 (UTC) Delivered-To: threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5ED1344C60 for ; Sun, 14 Jun 2020 21:00:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49lRgT548wz4sZB for ; Sun, 14 Jun 2020 21:00:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 95050126AE for ; Sun, 14 Jun 2020 21:00:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 05EL0LXk033351 for ; Sun, 14 Jun 2020 21:00:21 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 05EL0LbC033350 for threads@FreeBSD.org; Sun, 14 Jun 2020 21:00:21 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202006142100.05EL0LbC033350@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: threads@FreeBSD.org Subject: Problem reports for threads@FreeBSD.org that need special attention Date: Sun, 14 Jun 2020 21:00:21 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2020 21:00:22 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 237195 | pthread_mutex_unlock crash as unlocked mutex dest 1 problems total for which you should take action. From owner-freebsd-threads@freebsd.org Tue Jun 16 21:52:06 2020 Return-Path: Delivered-To: freebsd-threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 28E63336803 for ; Tue, 16 Jun 2020 21:52:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 49mhkG0N3tz3Y8Y for ; Tue, 16 Jun 2020 21:52:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 0CD49336779; Tue, 16 Jun 2020 21:52:06 +0000 (UTC) Delivered-To: threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C9C73362EE for ; Tue, 16 Jun 2020 21:52:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49mhkF6gxPz3Y3D for ; Tue, 16 Jun 2020 21:52:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C83161602C for ; Tue, 16 Jun 2020 21:52:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 05GLq5Iv069464 for ; Tue, 16 Jun 2020 21:52:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 05GLq57f069463 for threads@FreeBSD.org; Tue, 16 Jun 2020 21:52:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: threads@FreeBSD.org Subject: [Bug 244493] databases/lmdb: issue with MDB_USE_POSIX_MUTEX Date: Tue, 16 Jun 2020 21:52:05 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2020 21:52:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D244493 --- Comment #1 from commit-hook@freebsd.org --- A commit references this bug: Author: delphij Date: Tue Jun 16 21:51:56 UTC 2020 New revision: 539379 URL: https://svnweb.freebsd.org/changeset/ports/539379 Log: databases/lmdb: in db_env_close0(), destroy robust mutexes if we are the only remaining user. When closing an lmdb database, all memory and file descriptor resources are released, including the shared memory pages that contained the robust mutex. However, before this commit, prior to unmapping the pages that contained the robust mutexex, lmdb did not destroy the mutexes first. This would create a problem when an application opens and closes a database, then open it again. According to libthr(3), by default, a shared lock backed by a mapped file in memory is automatically destroyed on the last unmap of the corresponding file' page, which is allowed by POSIX. After unmapping the shared pages, the kernel writes off all active robust mutexes associated with these pages. However, the userland threading library still keeps the record (pshared_lookup in thr_pshared.c of libthr) for these objects as they are not really destroyed before, so that it don't have to ask the kernel every time when looking them up. Now, a later re-open of the database might have mapped the lock file to the same memory location. Because the threading library have remembered the robust mutex object, it would just reuse it even though it was already invalid from kernel's point of view. Unfortunately, regular lock operations would still work for this process. Should another lmdb process opens the same database, it would attempt to obtain the robust mutex (no longer recognized by kernel) because it would see another process holding a file lock, but that would fail because the robust mutex is invalid for the kernel. Explicitly destroy the mutex if we are the last remaining user to ensure the mutex is always in a known defined state. OpenLDAP ITS #9278 With debugging help from: kib PR: 244493 MFH: 2020Q2 Changes: head/databases/lmdb/Makefile head/databases/lmdb/files/patch-mdb.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-threads@freebsd.org Tue Jun 16 21:55:08 2020 Return-Path: Delivered-To: freebsd-threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 12ACE3366CA for ; Tue, 16 Jun 2020 21:55:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 49mhnl6lcTz3YH6 for ; Tue, 16 Jun 2020 21:55:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id E7899336914; Tue, 16 Jun 2020 21:55:07 +0000 (UTC) Delivered-To: threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E74F43367A9 for ; Tue, 16 Jun 2020 21:55:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49mhnl5wZfz3YH5 for ; Tue, 16 Jun 2020 21:55:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A98211603B for ; Tue, 16 Jun 2020 21:55:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 05GLt7W9071624 for ; Tue, 16 Jun 2020 21:55:07 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 05GLt7Oq071623 for threads@FreeBSD.org; Tue, 16 Jun 2020 21:55:07 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: threads@FreeBSD.org Subject: [Bug 244493] databases/lmdb: issue with MDB_USE_POSIX_MUTEX Date: Tue, 16 Jun 2020 21:55:07 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2020 21:55:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D244493 --- Comment #2 from commit-hook@freebsd.org --- A commit references this bug: Author: delphij Date: Tue Jun 16 21:54:59 UTC 2020 New revision: 539380 URL: https://svnweb.freebsd.org/changeset/ports/539380 Log: MFH: r539379 databases/lmdb: in db_env_close0(), destroy robust mutexes if we are the only remaining user. When closing an lmdb database, all memory and file descriptor resources are released, including the shared memory pages that contained the robust mutex. However, before this commit, prior to unmapping the pages that contained the robust mutexex, lmdb did not destroy the mutexes first. This would create a problem when an application opens and closes a database, then open it again. According to libthr(3), by default, a shared lock backed by a mapped file in memory is automatically destroyed on the last unmap of the corresponding file' page, which is allowed by POSIX. After unmapping the shared pages, the kernel writes off all active robust mutexes associated with these pages. However, the userland threading library still keeps the record (pshared_lookup in thr_pshared.c of libthr) for these objects as they are not really destroyed before, so that it don't have to ask the kernel every time when looking them up. Now, a later re-open of the database might have mapped the lock file to the same memory location. Because the threading library have remembered the robust mutex object, it would just reuse it even though it was already invalid from kernel's point of view. Unfortunately, regular lock operations would still work for this process. Should another lmdb process opens the same database, it would attempt to obtain the robust mutex (no longer recognized by kernel) because it would see another process holding a file lock, but that would fail because the robust mutex is invalid for the kernel. Explicitly destroy the mutex if we are the last remaining user to ensure the mutex is always in a known defined state. OpenLDAP ITS #9278 With debugging help from: kib PR: 244493 Approved by: ports-secteam Changes: _U branches/2020Q2/ branches/2020Q2/databases/lmdb/Makefile branches/2020Q2/databases/lmdb/files/patch-mdb.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-threads@freebsd.org Tue Jun 16 21:56:05 2020 Return-Path: Delivered-To: freebsd-threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4EF75336531 for ; Tue, 16 Jun 2020 21:56:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 49mhps1Q8Qz3YYQ for ; Tue, 16 Jun 2020 21:56:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 306903368B9; Tue, 16 Jun 2020 21:56:05 +0000 (UTC) Delivered-To: threads@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3035333652F for ; Tue, 16 Jun 2020 21:56:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49mhps0cFLz3YSc for ; Tue, 16 Jun 2020 21:56:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1022E15EC1 for ; Tue, 16 Jun 2020 21:56:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 05GLu4xE072291 for ; Tue, 16 Jun 2020 21:56:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 05GLu4vS072290 for threads@FreeBSD.org; Tue, 16 Jun 2020 21:56:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: threads@FreeBSD.org Subject: [Bug 244493] databases/lmdb: issue with MDB_USE_POSIX_MUTEX Date: Tue, 16 Jun 2020 21:56:05 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: delphij@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2020 21:56:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D244493 Xin LI changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |FIXED --=20 You are receiving this mail because: You are on the CC list for the bug.=