Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 28 Jun 2014 19:33:07 +0300
From:      Konstantin Belousov <kostikbel@gmail.com>
To:        Palle Girgensohn <girgen@pingpong.net>
Cc:        "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>, "performance@freebsd.org" <performance@freebsd.org>, John Baldwin <jhb@freebsd.org>
Subject:   Re: PostgreSQL performance on FreeBSD
Message-ID:  <20140628163307.GC93733@kib.kiev.ua>
In-Reply-To: <B97CF825-B9C3-4D46-A26C-7EF56CD5D216@pingpong.net>
References:  <20140627125613.GT93733@kib.kiev.ua> <201406271057.53599.jhb@freebsd.org> <20140627163407.GX93733@kib.kiev.ua> <35090A62-2DB8-493C-A5ED-ADB1BC193640@pingpong.net> <20140628102137.GA93733@kib.kiev.ua> <B97CF825-B9C3-4D46-A26C-7EF56CD5D216@pingpong.net>

next in thread | previous in thread | raw e-mail | index | archive | help

--xqJRvRgB/be8tbVJ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sat, Jun 28, 2014 at 01:37:20PM +0200, Palle Girgensohn wrote:
>=20
>=20
> > 28 jun 2014 kl. 12:21 skrev Konstantin Belousov <kostikbel@gmail.com>:
> >=20
> >> On Sat, Jun 28, 2014 at 12:08:39PM +0200, Palle Girgensohn wrote:
> >>=20
> >>=20
> >>> 27 jun 2014 kl. 18:34 skrev Konstantin Belousov <kostikbel@gmail.com>:
> >>>=20
> >>>> On Fri, Jun 27, 2014 at 10:57:53AM -0400, John Baldwin wrote:
> >>>>> On Friday, June 27, 2014 8:56:13 am Konstantin Belousov wrote:
> >>>>> Hi,
> >>>>> I did some measurements and hacks to see about the performance and
> >>>>> scalability of PostgreSQL 9.3 on FreeBSD, sponsored by The FreeBSD
> >>>>> Foundation.
> >>>>>=20
> >>>>> The results are described in https://kib.kiev.ua/kib/pgsql_perf.pdf.
> >>>>> The uncommitted patches, referenced in the article, are available as
> >>>>> https://kib.kiev.ua/kib/pig1.patch.txt
> >>>>> https://kib.kiev.ua/kib/patch-2
> >>>>=20
> >>>> Did you run the same benchmark on the same hardware with any other O=
S's to=20
> >>>> compare results?
> >>>=20
> >>> No.
> >>>=20
> >>> FWIW, before the failing after the 30 clients is corrected, I do not
> >>> think it is much interesting to do such comparision.
> >>=20
> >> This is great work!
> >>=20
> >> Does anybody know how far back in FreeBSD versions using posix semapho=
re instead of sysv would make a difference?  It seems we need a rather curr=
ent version? 8.x did not support it at all, at some point at lest, and in 9=
 it was buggy. I could add he patch-2 to the port, but I reckon it needs a =
conditional based on FreeBSD version?
> > I recommend to add it as an option.  The currently supported versions
> > of stable/9 and higher have new posix semaphores implementation.
> > The stable/8 also has posix semaphores, but there it is kernel-based
> > interface, I do not plan to evaluate it in any way.
>=20
> According to one source, posix semaphores uses O(N^2) file descriptors, w=
here N is the number of connections. Do you know if this is true? (I'll try=
 it, naturally, just checking).=20
>=20
(New) posix semaphores implementation, done by David Xu, opens a file
descriptor during the sem_open(3), which is used to mmap the area carrying
the lock word, and is immediately closed afterward in sem_open().  In
other words, if you have N semaphores and M processes, there would
be N*M open(2)/close(2) pairs, and N files, each mmaped to M processes.
New implementation does not use file descriptor during semaphore use,
and does not keep the backing file open.

> >=20
> >=20
> >> The clang bug should go upstreams, right?
> > I believe there is already some activity about it.  I do not follow
> > clang development.
>=20
> Sounds good enough.=20
>=20
> >=20
> >>=20
> >> I have seen similar curves, presented by Greg Smith (PostgreSQL
> >> hacker) where he concluded that there is no point in running more
> >> than 50 concurrent connections. This was for Linux. In your measures,
> >> the knee is at 30. That's said, FreeBSD could and should do better,
> >> but probably there is a limit where there will be a knee in the graph
> >> and performance will drop. It should be more than 30, though, as you
> >> rightly commented.
> >>=20
> >> Do you any ideas to pursue this further apart from complicated
> >> rewrites like DragonFly?
> > I do.
> >=20
> > The scope of the current work was done to obtain understanding where do=
 we stay
> > and, if possible, evaluate ideas, possibly in the hackish way. I hope
> > and almost sure that this will be continued, but cannot provide any time
> > estimation.
>=20
> Great. If you need help testing, I might be able to help.=20
I have the test set up and the graphing mostly automated, although
the repeat of the configuration would be quite laborous.

On the other hand, if you get access to zoo, replication could be easier.

--xqJRvRgB/be8tbVJ
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBAgAGBQJTru5DAAoJEJDCuSvBvK1B4R0P/2hZk6AgtyDcvVnrq135PNDn
JeGNzD2cMIa8rYVc3UMAUCar8VzFXPxgNSKJSKhEEEfonD/BS94f4J75vC6okUiO
b+5CidshF4KWElXAv6vzA5xnPverGzqeHpU6LRDGRotkgSUH2fgQ3EiR4g5a2kTC
4SZpRcUW708HoGHsvl0ivrBgQRIX0r6isXiUIJntG5OrbGUa/2VlVqlmIuR6Ckw5
+Vao8HAOL0JENuu1PPdzp0MWtZ0O+Klkt0vMcNN4SR7ePFHGT95OSAg+XG5rfSi+
7fRQmmT5MjimnArCP2nQkBVBvI4XRsvCoo2n02e+A2OkPRYJNGyY7pWzY39/OPtJ
cyfLkaxR3PidGSFxrU5N4YFQwgfsP/rBbC4+07Tskf6rGoRelUOmjbMkobyHB7JC
Gfa5QeTVNYL+6CbVjD4Y9o5FPof+FUHi2ul4kcGcJCtprfvrx9h9FjBa0NT2URrD
CMtdF7vA+/pVsJ6K22x7/QSVnpLc5tLpOWi8ULCdnliOANlZMWLZjci+EhhkC5sa
GX6duPXf8zSHifb+AaPwjF/XAGzD90zLBe2SEIAHMs6Q179LRBKN5N+ub/JyoVY8
5PMKpDbV2YCJMcyvkDXztDWJ0gv+C+OXeHywdwAW6kUOP3Pe/7713Yq8nERKC+Tk
8Epwei1gXgeP+0ZG7Bix
=YV3S
-----END PGP SIGNATURE-----

--xqJRvRgB/be8tbVJ--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140628163307.GC93733>