From owner-freebsd-stable@freebsd.org Mon Mar 7 02:19:50 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03A9EAC1203 for ; Mon, 7 Mar 2016 02:19:50 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from mail.samsco.org (suzy.samsco.org [168.103.85.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6134AC0 for ; Mon, 7 Mar 2016 02:19:49 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from localhost (localhost [192.168.254.3]) by mail.samsco.org (Postfix) with ESMTP id D0EF015C1867E; Mon, 7 Mar 2016 02:21:39 +0000 (UTC) Received: from mail.samsco.org ([192.168.254.3]) by localhost (mail.samsco.org [192.168.254.3]) (maiad, port 10024) with ESMTP id 74691-03; Mon, 7 Mar 2016 02:21:39 +0000 (UTC) Received: from [10.210.15.249] (c-24-130-241-190.hsd1.ca.comcast.net [24.130.241.190]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: scottl@samsco.org) by mail.samsco.org (Postfix) with ESMTPSA id 63F9515C1867C; Mon, 7 Mar 2016 02:21:39 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: kernel: mps0: Out of chain frames, consider increasing hw.mps.max_chains. From: Scott Long In-Reply-To: <20160306212733.GJ11654@zxy.spb.ru> Date: Sun, 6 Mar 2016 18:20:06 -0800 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20160306194555.GC94639@zxy.spb.ru> <0F0C78F4-6FE2-43BA-B503-AA04A79F2E70@samsco.org> <20160306212733.GJ11654@zxy.spb.ru> To: Slawa Olhovchenkov X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 02:19:50 -0000 > On Mar 6, 2016, at 1:27 PM, Slawa Olhovchenkov wrote: >=20 > On Sun, Mar 06, 2016 at 01:10:42PM -0800, Scott Long wrote: >=20 >> Hi, >>=20 >> The message is harmless, it's a reminder that you should tune the = kernel for your workload. When the message is triggered, it means that = a potential command was deferred, likely for only a few microseconds, = and then everything moved on as normal. =20 >>=20 >> A command uses anywhere from 0 to a few dozen chain frames per I/o, = depending on the size of the io. The chain frame memory is allocated at = boot so that it's always available, not allocated on the fly. When I = wrote this driver, I felt that it would be wasteful to reserve memory = for a worst case scenario of all large io's by default, so I put in this = deferral system with a console reminder to for tuning. =20 >>=20 >> Yes, you actually do have 900 io's outstanding. The controller = buffers the io requests and allows the system to queue up much more than = what sata disks might allow on their own. It's debatable if this is = good or bad, but it's tunable as well. >>=20 >> Anyways, the messages should not cause alarm. Either tune up the = chain frame count, or tune down the max io count. >=20 > I am don't know depends or not, but I see dramaticaly performance drop > at time of this messages. >=20 Good to know. Part of the performance drop might be because of the = slowness of printing to the console. > How I can calculate buffers numbers? If your system is new enough to have mpsutil, please run it =E2=80=98mpsut= il show iocfacts=E2=80=99. If not, then boot your system with = bootverbose and send me the output. > I am have very heavy I/O. Out of curiosity, do you redefine MAXPHYS/DFLTPHYS in your kernel = config? > This allocated one for all controllers, or allocated for every = controller? It=E2=80=99s per-controller. I=E2=80=99ve thought about making the tuning be dynamic at runtime. I = implemented similar dynamic tuning for other drivers, but it seemed = overly complex for low benefit. Implementing it for this driver would = be possible but require some significant code changes. Scott