From owner-freebsd-ports-bugs@freebsd.org Fri Feb 28 10:48:42 2020 Return-Path: Delivered-To: freebsd-ports-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F2A3425FF36 for ; Fri, 28 Feb 2020 10:48:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48TR96646Xz43r1 for ; Fri, 28 Feb 2020 10:48:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id CE18E25FF35; Fri, 28 Feb 2020 10:48:42 +0000 (UTC) Delivered-To: ports-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CCCB125FF34 for ; Fri, 28 Feb 2020 10:48:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TR964zzBz43qv for ; Fri, 28 Feb 2020 10:48:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A592B5D9 for ; Fri, 28 Feb 2020 10:48:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 01SAmgdR013289 for ; Fri, 28 Feb 2020 10:48:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 01SAmg22013288 for ports-bugs@FreeBSD.org; Fri, 28 Feb 2020 10:48:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: ports-bugs@FreeBSD.org Subject: [Bug 244493] databases/lmdb: issue with MDB_USE_POSIX_MUTEX Date: Fri, 28 Feb 2020 10:48:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: freebsd@kempniu.pl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter flagtypes.name attachments.mimetype attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ports-bugs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Ports bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 10:48:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D244493 Bug ID: 244493 Summary: databases/lmdb: issue with MDB_USE_POSIX_MUTEX Product: Ports & Packages Version: Latest Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: Individual Port(s) Assignee: delphij@FreeBSD.org Reporter: freebsd@kempniu.pl Assignee: delphij@FreeBSD.org Flags: maintainer-feedback?(delphij@FreeBSD.org) Attachment #212019 text/plain mime type: Created attachment 212019 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D212019&action= =3Dedit Sample program reproducing the issue Hi there, Ports r519246 switched LMDB from MDB_USE_POSIX_SEM to MDB_USE_POSIX_MUTEX. Unfortunately, it seems that there are some edge cases in which LMDB does not play nicely with FreeBSD's process-shared mutexes. The particular problem I observed is that when a single process reopens an LMDB environment (that is, an environment is opened, closed, then opened again by the same process), then other processes trying to access the reopened environment fail to grab the read table lock - pthread_mutex_lock() returns EINVAL (22). AFAICT, this happens because libthr is unable to find the shared memory segment with the relevant part of the LMDB lockfile mmap()'d. I attached the simplest test case I could come up with. To reproduce the problem, first compile lmdb-mutex.c: cc -I/usr/local/include -L/usr/local/lib -llmdb lmdb-mutex.c -o lmdb-mu= tex Then, run the first instance of the program. It should start fine and sleep for 30 seconds. Before the first instance exits, start a second instance of the program. It should fail with: Assertion failed: (mdb_txn_begin(env, 0, MDB_RDONLY, &txn) =3D=3D MDB_S= UCCESS), function main, file lmdb-mutex.c, line 14. The sample program works fine in the above scenario if LMDB is compiled with MDB_USE_POSIX_SEM. It also works fine on other operating systems using MDB_USE_POSIX_MUTEX. One workaround I could come up with is enabling ASLR - it causes mmap() to return different addresses for the LMDB lockfile mapping upon each call to mdb_env_open(), causing libthr to use different off-pages for the read table mutexes for the "old" and "new" environment (IIUC). Note that LMDB never calls pthread_mutex_destroy() for the read table lock when an environment is closed which I believe prevents the shared memory segment for the "old" environment from being released. But please do not take my word for it, I do not understand sys/kern/kern_umtx.c, libthr, and LMDB internals well enough to fully explain what is happening (though I sure would like to find out!) To see an example occurrence of this problem in the wild, install BIND (e.g. dns/bind911), put the following into named.conf: options { allow-new-zones yes; }; and then run "named -g -c named.conf". After it starts up, run "named-nzd2nzf _default.nzd". It will fail with: named-nzd2nzf: mdb_txn_begin: Invalid argument My humble suggestion would be to revert the LMDB port back to MDB_USE_POSIX_SEM for the time being, unless someone can immediately see what the problem is and is able to fix it. Hope this helps, please let me know if I can be of any further assistance with this issue. --=20 You are receiving this mail because: You are the assignee for the bug.=