From owner-freebsd-performance@freebsd.org Fri Jun 24 00:27:29 2016 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4292DAC6E7C for ; Fri, 24 Jun 2016 00:27:29 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 252562F33 for ; Fri, 24 Jun 2016 00:27:29 +0000 (UTC) (envelope-from sean@chittenden.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1E449AC6E79; Fri, 24 Jun 2016 00:27:29 +0000 (UTC) Delivered-To: performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18C63AC6E77; Fri, 24 Jun 2016 00:27:29 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE5432F2C; Fri, 24 Jun 2016 00:27:28 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from [172.20.10.9] (unknown [172.56.39.215]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id 1E1BD2F88F5; Thu, 23 Jun 2016 17:19:15 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: PostgreSQL performance on FreeBSD From: Sean Chittenden In-Reply-To: Date: Thu, 23 Jun 2016 15:42:32 -0700 Cc: Konstantin Belousov , Adrian Chadd , performance@freebsd.org, John Baldwin , Alan Somers , Alan Cox , Alan Cox , freebsd-current , "current@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140627125613.GT93733@kib.kiev.ua> <1603235.2ShtoCfSqO@ralph.baldwin.cx> <20160622100241.GM38613@kib.kiev.ua> To: Maxim Sobolev X-Mailer: Apple Mail (2.3124) X-Mailman-Approved-At: Fri, 24 Jun 2016 01:26:02 +0000 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jun 2016 00:27:29 -0000 Small nit: PostgreSQL used SYSV because it allowed for the detection of dead = processes. If you `kill -9`=E2=80=99ed a process, PostgreSQL can detect = that and then shut down and perform an automatic recovery. In this = regard, sysv is pretty clever. The move to POSIX shared mem was done = for a host of reasons, but it means that you don=E2=80=99t have to = adjust your SYSV limits. My understanding from a few years ago is that = there is still a ~64KB SYSV memory segment that is still used to act as = the latch to signal if a process was killed, but all of the shared = buffers are stored in posix mmap=E2=80=99ed regions. At this point in time this could be replaced with kqueue(2) EVFILT_PROC, = but no one has done that yet. -sc -- Sean Chittenden sean@chittenden.org > On Jun 22, 2016, at 07:26 , Maxim Sobolev wrote: >=20 > Konstantin, >=20 > Not if you do sem_unlink() immediately, AFAIK. And that's what PG = does. So > the window of opportunity for the leakage is quite small, much smaller = than > for SYSV primitives. Sorry for missing your status update message, = I've > missed it somehow. >=20 > ---- > mySem =3D sem_open(semname, O_CREAT | O_EXCL, > (mode_t) = IPCProtection, > (unsigned) 1); >=20 > #ifdef SEM_FAILED > if (mySem !=3D (sem_t *) SEM_FAILED) > break; > #else > if (mySem !=3D (sem_t *) (-1)) > break; > #endif >=20 > /* Loop if error indicates a collision */ > if (errno =3D=3D EEXIST || errno =3D=3D EACCES || errno = =3D=3D EINTR) > continue; >=20 > /* > * Else complain and abort > */ > elog(FATAL, "sem_open(\"%s\") failed: %m", semname); > } >=20 > /* > * Unlink the semaphore immediately, so it can't be accessed > externally. > * This also ensures that it will go away if we crash. > */ > sem_unlink(semname); >=20 > return mySem; > ---- >=20 > -Max >=20 > On Wed, Jun 22, 2016 at 3:02 AM, Konstantin Belousov = > wrote: >=20 >> On Tue, Jun 21, 2016 at 12:48:00PM -0700, Maxim Sobolev wrote: >>> Thanks, Konstantin for the great work, we are definitely looking = forward >> to >>> get all those improvements to be part of the default FreeBSD = kernel/port. >>> Would be nice if you can post an update some day later as to what's >>> integrated and what's not. >> I did posted the update several days earlier. Since you replying to = this >> thread, it would be not unreasonable to read recent messages that = were >> sent. >>=20 >>>=20 >>> Just in case, I've opened #14206 with PG to switch us to using POSIX >>> semaphores by default. Apart from the mentioned performance = benefits, >> SYSV >>> semaphores are PITA to deal with as they come in very limited = quantities >> by >>> default. Also they might stay around if PG dies/gets nuked and = prevent it >>> from starting again due to overflow. We've got some quite ugly code = to >>> clean up those using ipcrm(1) in our build scripts to deal with just >> that. >>> I am happy that code could be retired now. >>=20 >> Named semaphores also stuck around if processes are killed without = cleanup. >>=20 >>=20 > _______________________________________________ > freebsd-performance@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to = "freebsd-performance-unsubscribe@freebsd.org"