From owner-svn-src-all@freebsd.org Tue Mar 12 17:21:12 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C13BF15378D9; Tue, 12 Mar 2019 17:21:12 +0000 (UTC) (envelope-from freebsd@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 149FB726B7; Tue, 12 Mar 2019 17:21:11 +0000 (UTC) (envelope-from freebsd@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x2CHL92P016000; Tue, 12 Mar 2019 10:21:09 -0700 (PDT) (envelope-from freebsd@gndrsh.dnsmgr.net) Received: (from freebsd@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x2CHL9O2015999; Tue, 12 Mar 2019 10:21:09 -0700 (PDT) (envelope-from freebsd) From: "Rodney W. Grimes" Message-Id: <201903121721.x2CHL9O2015999@gndrsh.dnsmgr.net> Subject: Re: svn commit: r345056 - head/sys/dev/mrsas In-Reply-To: <201903120924.x2C9Ow52017475@repo.freebsd.org> To: Kashyap D Desai Date: Tue, 12 Mar 2019 10:21:09 -0700 (PDT) CC: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Reply-To: rgrimes@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 149FB726B7 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.958,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2019 17:21:12 -0000 > Author: kadesai > Date: Tue Mar 12 09:24:58 2019 > New Revision: 345056 > URL: https://svnweb.freebsd.org/changeset/base/345056 > > Log: > fw_outstanding"(outstanding IOs at firmware level) counter gets screwed up when R1 fastpath > writes are running. Some of the cases which are not handled properly in driver are: > > 1. With R1 fastpath supported, single write from CAM layer can consume 2 MPT frames > at driver/firmware level for fastpath qualification(if fw_outstanding < controller Queue Depth). > Due to this driver has to throttle IOs coming from CAM layer as well as second fastpath > write(of R1 write) against Adapter Queue Depth. > If "fw_outstanding" reaches to adapter queue depth, driver should return IOs from CAM layer with > device busy status.While allocating second MPT frame(corresponding to R1 FP write) also, driver > should ensure fw_outstanding should not exceed adapter QD. > > 2. For R1 fastpath writes completion, driver decrements "fw_oustanding" counter without > really returning MPT frame to free pool. It may cause IOs(with heavy IOs running, consuming whole > adapter Queue Depth) consuming MPT frames reserved for DCMDs(management commands) and > DCMDs(internal and sent by application) not getting MPT frame will start failing. > > Below is one test case to hit the issue described above- > 1. Run heavy IOs (outstanding IOs should hit adapter Queue Depth). > 2. Run management tool (Broadcom's storcli tool) querying adapter in loop (run command- "storcli64 /c0 show" in loop). > 3. Management tool's requests would start failing due to non-availability of free MPT frames as all frames would be consumed by IOs. > > Fix: Increment/decrement of "fw_outstanding" counter should be in sync with MPT frame get/return. > > Submitted by: Sumit Saxena > Reviewed by: Kashyap Desai > Approved by: Ken > MFC after: 3 days > Sponsored by: Broadcom Inc > Can we please keep commit messages formatted to <80 columns? Thanks, Rod -- Rod Grimes rgrimes@freebsd.org