From owner-svn-ports-all@freebsd.org Tue Jun 16 21:54:59 2020 Return-Path: Delivered-To: svn-ports-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 90A693366C4; Tue, 16 Jun 2020 21:54:59 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49mhnb3L7Bz3Xs1; Tue, 16 Jun 2020 21:54:59 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6D94A16039; Tue, 16 Jun 2020 21:54:59 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05GLsxvm008097; Tue, 16 Jun 2020 21:54:59 GMT (envelope-from delphij@FreeBSD.org) Received: (from delphij@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05GLsxYg008095; Tue, 16 Jun 2020 21:54:59 GMT (envelope-from delphij@FreeBSD.org) Message-Id: <202006162154.05GLsxYg008095@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: delphij set sender to delphij@FreeBSD.org using -f From: Xin LI Date: Tue, 16 Jun 2020 21:54:59 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-branches@freebsd.org Subject: svn commit: r539380 - in branches/2020Q2/databases/lmdb: . files X-SVN-Group: ports-branches X-SVN-Commit-Author: delphij X-SVN-Commit-Paths: in branches/2020Q2/databases/lmdb: . files X-SVN-Commit-Revision: 539380 X-SVN-Commit-Repository: ports MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-ports-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the ports tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2020 21:54:59 -0000 Author: delphij Date: Tue Jun 16 21:54:58 2020 New Revision: 539380 URL: https://svnweb.freebsd.org/changeset/ports/539380 Log: MFH: r539379 databases/lmdb: in db_env_close0(), destroy robust mutexes if we are the only remaining user. When closing an lmdb database, all memory and file descriptor resources are released, including the shared memory pages that contained the robust mutex. However, before this commit, prior to unmapping the pages that contained the robust mutexex, lmdb did not destroy the mutexes first. This would create a problem when an application opens and closes a database, then open it again. According to libthr(3), by default, a shared lock backed by a mapped file in memory is automatically destroyed on the last unmap of the corresponding file' page, which is allowed by POSIX. After unmapping the shared pages, the kernel writes off all active robust mutexes associated with these pages. However, the userland threading library still keeps the record (pshared_lookup in thr_pshared.c of libthr) for these objects as they are not really destroyed before, so that it don't have to ask the kernel every time when looking them up. Now, a later re-open of the database might have mapped the lock file to the same memory location. Because the threading library have remembered the robust mutex object, it would just reuse it even though it was already invalid from kernel's point of view. Unfortunately, regular lock operations would still work for this process. Should another lmdb process opens the same database, it would attempt to obtain the robust mutex (no longer recognized by kernel) because it would see another process holding a file lock, but that would fail because the robust mutex is invalid for the kernel. Explicitly destroy the mutex if we are the last remaining user to ensure the mutex is always in a known defined state. OpenLDAP ITS #9278 With debugging help from: kib PR: 244493 Approved by: ports-secteam Modified: branches/2020Q2/databases/lmdb/Makefile branches/2020Q2/databases/lmdb/files/patch-mdb.c Directory Properties: branches/2020Q2/ (props changed) Modified: branches/2020Q2/databases/lmdb/Makefile ============================================================================== --- branches/2020Q2/databases/lmdb/Makefile Tue Jun 16 21:51:55 2020 (r539379) +++ branches/2020Q2/databases/lmdb/Makefile Tue Jun 16 21:54:58 2020 (r539380) @@ -3,7 +3,7 @@ PORTNAME= lmdb PORTVERSION= 0.9.24 -PORTREVISION= 1 +PORTREVISION= 2 DISTVERSIONPREFIX= ${PORTNAME:tu}_ PORTEPOCH= 1 CATEGORIES= databases Modified: branches/2020Q2/databases/lmdb/files/patch-mdb.c ============================================================================== --- branches/2020Q2/databases/lmdb/files/patch-mdb.c Tue Jun 16 21:51:55 2020 (r539379) +++ branches/2020Q2/databases/lmdb/files/patch-mdb.c Tue Jun 16 21:54:58 2020 (r539380) @@ -5,10 +5,37 @@ #endif -#if defined(__APPLE__) || defined (BSD) || defined(__FreeBSD_kernel__) -+#if defined(__FreeBSD__) -+# define MDB_USE_POSIX_MUTEX 1 -+# define MDB_USE_ROBUST 1 ++#if defined(__FreeBSD__) && defined(__FreeBSD_version) && __FreeBSD_version >= 1100110 ++# define MDB_USE_POSIX_MUTEX 1 ++# define MDB_USE_ROBUST 1 +#elif defined(__APPLE__) || defined (BSD) || defined(__FreeBSD_kernel__) # define MDB_USE_POSIX_SEM 1 # define MDB_FDATASYNC fsync #elif defined(ANDROID) +@@ -1375,7 +1378,7 @@ static int mdb_page_split(MDB_cursor *mc, MDB_val *new + static int mdb_env_read_header(MDB_env *env, MDB_meta *meta); + static MDB_meta *mdb_env_pick_meta(const MDB_env *env); + static int mdb_env_write_meta(MDB_txn *txn); +-#ifdef MDB_USE_POSIX_MUTEX /* Drop unused excl arg */ ++#if defined(MDB_USE_POSIX_MUTEX) && !defined(MDB_ROBUST_SUPPORTED) /* Drop unused excl arg */ + # define mdb_env_close0(env, excl) mdb_env_close1(env) + #endif + static void mdb_env_close0(MDB_env *env, int excl); +@@ -5127,6 +5130,17 @@ mdb_env_close0(MDB_env *env, int excl) + sem_unlink(env->me_txns->mti_rmname); + sem_unlink(env->me_txns->mti_wmname); + } ++ } ++#elif defined(MDB_ROBUST_SUPPORTED) ++ /* If we have the filelock: If we are the ++ * only remaining user, clean up robust ++ * mutexes. ++ */ ++ if (excl == 0) ++ mdb_env_excl_lock(env, &excl); ++ if (excl > 0) { ++ pthread_mutex_destroy(env->me_txns->mti_rmutex); ++ pthread_mutex_destroy(env->me_txns->mti_wmutex); + } + #endif + munmap((void *)env->me_txns, (env->me_maxreaders-1)*sizeof(MDB_reader)+sizeof(MDB_txninfo));