From owner-freebsd-stable@FreeBSD.ORG Tue Aug 14 17:27:04 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDD0516A469 for ; Tue, 14 Aug 2007 17:27:04 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 6013213C4B7 for ; Tue, 14 Aug 2007 17:27:04 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7EHQxx8088205; Tue, 14 Aug 2007 11:27:00 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46C1E5E0.1080508@samsco.org> Date: Tue, 14 Aug 2007 11:26:56 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: Howard Goldstein References: <46C0A495.4060706@queue.to> <46C0A812.7080702@samsco.org> <46C1E49C.20701@queue.to> In-Reply-To: <46C1E49C.20701@queue.to> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Tue, 14 Aug 2007 11:27:00 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-stable@freebsd.org Subject: Re: 3ware RAID ctrlr: twe0: twe_map_request: malloc failed. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2007 17:27:04 -0000 Howard Goldstein wrote: > Scott Long wrote: >> Howard Goldstein wrote: >>> Would someone confirm (or disabuse!) me of the belief that these 3ware >>> Escalade 8106-LP2 (2 port sata raid controllers) related messages are >>> informational and the UFS device is retrying after it sees ENOMEM >>> causing these? >>> >>> Aug 13 10:29:48 cally kernel: twe0: twe_map_request: malloc failed >>> Aug 13 10:33:04 cally kernel: twe0: twe_map_request: malloc failed >>> Aug 13 11:05:37 cally kernel: twe0: twe_map_request: malloc failed >>> Aug 13 11:14:55 cally kernel: twe0: twe_map_request: malloc failed >>> Aug 13 12:07:44 cally kernel: twe0: twe_map_request: malloc failed >>> Aug 13 12:28:05 cally kernel: twe0: twe_map_request: malloc failed >>> Aug 13 14:29:59 cally kernel: twe0: twe_map_request: malloc failed >>> >>> (Either way but particularly in the event the data are being lost is >>> there a sysctl or kernel build option to supply a bit more memory in >>> advance?) >> The system is doing unaligned I/O to the array (either via an app that >> has a file opened O_DIRECT, or via I/O directly to the device node), and >> there is corresponding memory pressure that is making the driver fail >> at the re-alignment process. The problem is likely temporary and the >> operation eventually succeeds after a few retries, but success is not >> guaranteed, and failure will almost certainly cause system instability. >> The fix is to rewrite the alignment code in the driver to use the >> automatic, failure-free, alignment service that busdma provides. I'm >> happy to provide direction to anyone who wants to tackle this. > > Alright, I like this idea since rebuilding the devices to avoid what I > thought were unaligned access didn't work and I'm still seeing these > messages and a performance hit I don't understand. I imagine I'll get > better performance too if the driver can avoid doing its own alignment > which probably isn't as optmized (?) The busdma man page looks nice and > complete too - cool > > Could you point me to a model driver or two already using the busdma stuff? > Sorry for the confusion, I'm talking about aligned memory accesses, not disk accesses. The TWE architecture requires that all transfers be on 4KB boundaries in memory. I'll write up some details on the work tonight. Basically it involves removing the manual aligment code and modifying the bus_dma tags to specify 4k alignment. It sounds easy, but I seem to recall something nasty in the code the last time I looked at it. Scott