From owner-freebsd-current@FreeBSD.ORG Wed Apr 8 02:36:06 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2813FC8D for ; Wed, 8 Apr 2015 02:36:06 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD2CF6DF for ; Wed, 8 Apr 2015 02:36:05 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t382ZxL9012734 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 8 Apr 2015 05:35:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t382ZxL9012734 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t382Zx0K012731; Wed, 8 Apr 2015 05:35:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 8 Apr 2015 05:35:59 +0300 From: Konstantin Belousov To: Garrett Cooper Subject: Re: mfi timeouts at boot on ASUS Z97 motherboard with LSI 9240-4i Message-ID: <20150408023559.GN2379@kib.kiev.ua> References: <64CF176D-910E-417D-98E2-9B2C64521444@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <64CF176D-910E-417D-98E2-9B2C64521444@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 02:36:06 -0000 On Tue, Apr 07, 2015 at 12:04:44PM -0700, Garrett Cooper wrote: > Hi, > I just tried to upgrade my system from a Nehalem style era CPU/motherboard (a W series Xeon/P6T-WS) to a Haswell style CPU/motherboard (an E-series Haswell/Z97 motherboard), and I?ve run into some fun issues with my LSI 9240-4i controller. In particular, it?s timing out because of issues noted similar to here: https://lists.freebsd.org/pipermail/freebsd-stable/2011-October/064404.html . I?ve done the following so far to try and diagnose the issue: > - Upgraded the firmware on the card and in the BIOS > - Tweaked with the PCI-E settings (2.0/3.0; disabled power savings mode in the BIOS; etc) > - Turned off 4GB ?remapping? on the PCM. > - Booted USB key fob-based images running: 9.3-RELEASE, 10.1-RELEASE, 10-STABLE, 11-CURRENT > - Booted my original drive running 9.3-RELEASE > - Booted with hw.mfi.msi=0 set in loader. > - Booted with boot -v. One more thing to try, is to put the card into PCIe slot handled by the south bridge instead of the CPU slot. I have two Drake Skinnies card which I cannot use, since all my machines are desktop class, and the only available PCIe 8x+ slots are connected to CPU. I was never able to get a satisfactory explanation why the thing does not like CPU' PCIe. > Every time I run into MFI send command timeouts that eventually turn into > Here?s the message I got (transcribed) with 11-CURRENT with boot -v: > > mfi0: port 0xe000-0xe0ff mem 0xdfc60000-0x?. on pci2 > mfi0: attempting to allocate 1 MSI vectors (1 supported) > pci: routing MSI IRQ 264 to local APIC 0 vector 59 > mfi0: using IRQ 264 for MSI > mfi0: using MSI > mfi0: Megaraid SAS driver Ver 4.23 > mfi0: Frame 0xfffffe07f719f280 timed out command 0x1010000 > Error 60 in callback from mfi_send_frame > > The complete screenshot for that timeout can be found here: https://people.freebsd.org/~ngie/mfi-timeout-boot-verbose.jpg > What I haven?t done yet: > - Exporting the volume with the old motherboard and importing it with the new motherboard (I need to do some reading before I do this though because I don?t want to toast my data by accident :(?). > - Booted either Linux or Windows to make sure the controller works with the motherboard > - Tried clearing events with the controller > Does anyone have any thoughts or hints on this issue? > Thanks!