Date: Fri, 17 Jun 2022 17:03:04 +0200 From: "Dave Cottlehuber" <dch@skunkwerks.at> To: "Mike Jakubik" <mike.jakubik@swiftsmsgateway.com>, freebsd-net <freebsd-net@FreeBSD.org> Cc: "Alexander V. Chernikov" <melifaro@ipfw.ru> Subject: Re: Poor performance with stable/13 and Mellanox ConnectX-6 (mlx5) Message-ID: <9892d8f8-df5d-4e73-ab48-cb5909b4f9c6@www.fastmail.com> In-Reply-To: <1816f19416a.b852ce5b2189187.4131912798685804323@swiftsmsgateway.com> References: <1815e506878.cf301a5a1195924.6506017618978817828@swiftsmsgateway.com> <63396d47-3d0b-fd83-7b2e-ae5c02eeae2e@selasky.org> <18162979a8f.e81f383a1466900.9104319828015733292@swiftsmsgateway.com> <18162a4a3f6.10a1a03d11472072.3783895140221599214@swiftsmsgateway.com> <1816e469bdf.126cdb81b2139485.369352368493375815@swiftsmsgateway.com> <C54B2F28-D02F-4589-82D7-1C84705C2C23@ipfw.ru> <1816f19416a.b852ce5b2189187.4131912798685804323@swiftsmsgateway.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 17 Jun 2022, at 02:38, Mike Jakubik wrote: > Hi, > > I believe you hit the nail on the head! I am now getting consistent=20 > high speeds, even higher than on Linux! Is this a problem with the=20 > scheduler? Should someone in that area of expertise be made aware of=20 > this? More importantly i guess, would this affect real world=20 > performance, these servers will be running RabbitMQ (it uses quite a=20 > bit of bandwidth) and PostgreSQL w/ replication. pinning cores for unimpeded access is very common for high performance s= ystems. Do this both for the nics and also your apps. Be mindful of the = NUMA topooogy. You should look into both the erlang scheduler flags for core pinning, = and also ensuring that your erlang processes have unimpeded access to th= eir own cores too. A reasonable approach is to make a simple cowboy or P= hoenix app and hammer it with wrk or similar load tool to get a feel for= things, and then profile and tune your own app based on those specific = results. For rabbit there is an excellent load testing tool from the pivotal team= if you don=E2=80=99t have suitable load generators yourselves. Tsung is an excellent tool if you put in the work to craft something spe= cific for your use case. Please post back to the list with your specific findings and nic/ tcp tu= nables, these are very helpful for the next person! Dave=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9892d8f8-df5d-4e73-ab48-cb5909b4f9c6>