From owner-freebsd-current@freebsd.org Thu Jun 11 15:19:34 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E52963355F5 for ; Thu, 11 Jun 2020 15:19:34 +0000 (UTC) (envelope-from SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 49jSFf4Xqkz3bdh for ; Thu, 11 Jun 2020 15:19:34 +0000 (UTC) (envelope-from SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net) Received: by mailman.nyi.freebsd.org (Postfix) id 9A06D3355F2; Thu, 11 Jun 2020 15:19:34 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 99C153357D4 for ; Thu, 11 Jun 2020 15:19:34 +0000 (UTC) (envelope-from SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net) Received: from bosmailout08.eigbox.net (bosmailout08.eigbox.net [66.96.187.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49jSFd5B3Bz3bSK; Thu, 11 Jun 2020 15:19:33 +0000 (UTC) (envelope-from SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net) Received: from bosmailscan14.eigbox.net ([10.20.15.14]) by bosmailout08.eigbox.net with esmtp (Exim) id 1jjOzN-0000bs-2G; Thu, 11 Jun 2020 11:19:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codenetworks.net; s=dkim; h=Sender:Content-Type:In-Reply-To:MIME-Version: Date:Message-ID:References:Cc:To:From:Subject:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=vhHXiDQ0+25TTbDPHaqt0hemIGvu7qIdZxeY+JWNdyg=; b=rsrTAN0wWjA3gNKRQu0PThgXq mTZjEKACnRKxhzOyWzZJeKAwO5eRj3+RFdnxzh2vgLiQWiWwJyWU0iemhynUnVqh6D5h+aUtZhANs +ncf9jvgJLUKc8Rpzi99ReoXef1cUN0LW77qM6msnPAJI+DyW5kPk1/8/6aaEfTmRghEua6pLC+uV VhUjv0dE2XYlb0BUZw0PUl3lbz9PJSd7S6Gcvv+pDh6pFtQ0EElgU2Fg4LgT4yqmRKIyEWFMdwEGR VWta5wq8OVi0/FGcqS5yZhFsxF9tF//BFbZdZ7GkOFUPofXk/9K7s8TctL+1/MwQtxYa5z9oELnXg Qvvo7velA==; Received: from [10.115.3.31] (helo=bosimpout11) by bosmailscan14.eigbox.net with esmtp (Exim) id 1jjOzN-0000Ka-0N; Thu, 11 Jun 2020 11:19:33 -0400 Received: from bosauthsmtp04.yourhostingaccount.com ([10.20.18.4]) by bosimpout11 with id prKV2200B05GATN01rKYYs; Thu, 11 Jun 2020 11:19:32 -0400 X-Authority-Analysis: v=2.2 cv=RKvDJ8q+ c=1 sm=1 tr=0 a=jcMFiYeD4Gz02xRQ37rMPA==:117 a=kkwQLKMA1cIZLGAtbCrT4w==:17 a=nTHF0DUjJn0A:10 a=zXyMKnP1Z5YA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=jlvWEfeLAAAA:8 a=Q-fNiiVtAAAA:8 a=6I5d2MoRAAAA:8 a=Mr1XnJQWAAAA:8 a=hwDfw3MnIapnvPP-XssA:9 a=TkbK1KLyaUlKGrww:21 a=NjtNdUuQBdQ50ap1:21 a=QEXdDO2ut3YA:10 a=_r76nwpHK7PlnpNLVdkA:9 a=gk9z3pULacAhocEm:21 a=mirLA6hOlVaX2wMo:21 a=P0A2uI_XTCv9nES3:21 a=_W_S_7VecoQA:10 a=BUduvz6nQKmfCEOu4uBS:22 a=Fp8MccfUoT0GBdDC_Lng:22 a=IjZwj45LgO3ly-622nXo:22 a=x1WslF1p1lK7vEzEGcse:22 Received: from cpc96142-rdng26-2-0-cust93.15-3.cable.virginm.net ([80.5.170.94]:21393 helo=[192.168.0.100]) by bosauthsmtp04.eigbox.net with esmtpa (Exim) id 1jjOzJ-0000Er-53; Thu, 11 Jun 2020 11:19:29 -0400 Subject: Re: MRSAS Panic during Install. From: Santiago Martinez To: Kashyap Desai , Don Lewis , Andriy Gapon Cc: FreeBSD Current , "Kashyap D. Desai" , "Kenneth D. Merry" , Sumit Saxena , Chandrakanth Patil References: <99447152-0340-11dd-fb75-a745d2ee5a6a@codenetworks.net> <0fa770fc-4fbc-cd1f-e8cc-30c74b37b71b@codenetworks.net> <3b0d4cec-69bd-0516-77e8-574678b6bd36@FreeBSD.org> <692229ea-ceb5-9e0d-e12e-2999ecb69297@codenetworks.net> <165ad464ab307e604a992dc59df40853@mail.gmail.com> <28f0a9f2-3f00-0167-bb5d-eab7d66ff813@codenetworks.net> <75ff0cfe-e516-a10a-abc5-7e1eb708c87f@codenetworks.net> Message-ID: <51abdead-b727-c3e3-b6a9-dfaaf400a6d8@codenetworks.net> Date: Thu, 11 Jun 2020 16:19:18 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-EN-UserInfo: d3bdfab0736480cedf04ed92aaea2ef5:931c98230c6409dcc37fa7e93b490c27 X-EN-AuthUser: sm@codenetworks.net Sender: Santiago Martinez X-EN-OrigIP: 80.5.170.94 X-EN-OrigHost: cpc96142-rdng26-2-0-cust93.15-3.cable.virginm.net X-Rspamd-Queue-Id: 49jSFd5B3Bz3bSK X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none (invalid DKIM record) header.d=codenetworks.net header.s=dkim header.b=rsrTAN0w; dmarc=none; spf=pass (mx1.freebsd.org: domain of SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net designates 66.96.187.8 as permitted sender) smtp.mailfrom=SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net X-Spamd-Result: default: False [-2.76 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-1.02)[-1.016]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.96.128.0/18]; NEURAL_HAM_LONG(-0.98)[-0.981]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[codenetworks.net: no valid DMARC record]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[codenetworks.net:~]; NEURAL_HAM_SHORT(-0.76)[-0.760]; RCPT_COUNT_SEVEN(0.00)[8]; RCVD_IN_DNSWL_NONE(0.00)[66.96.187.8:from]; R_DKIM_PERMFAIL(0.00)[codenetworks.net:s=dkim]; FORGED_SENDER(0.30)[sm@codenetworks.net,SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:29873, ipnet:66.96.128.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[sm@codenetworks.net,SRS0=uA56ax=7Y=codenetworks.net=sm@eigbox.net]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jun 2020 15:19:35 -0000 Hi Everyone,  i haven't forget about this.... yet :) I perform the following test in preparation to apply the patch: 1. Installed 12.1 stable with one disc in raid0, no issues (did this so i can apply patch and don't have to build a new bootable image). 2. with 12.1 added second raid, in this case RAID1 and it crashed. during foot complain about wrong/corrupt partition table Repeated the same test with current with exactly the same results ( I also hit another issues on mlx5en that crashed with current, will follow after we can sort this out) Today I'm planning to apply the patch and see what happens. Have some question: * Do you want me to record the booting/error for each case and make it available? * Do you want access to the box? Best regards. Santiago On 2020-06-09 22:43, Santiago Martinez wrote: > Hi there, apologies for the delayed response. > > Regarding the lock reversal, I can try to capture the screen showing > the message. > > The "Wil check go it goes...." it was my brain trying to multitask, > obviously not in a successful way. What I meant to say was "I will > check how it goes..... without the RAID". > > Sure, I will test with the patch and let you know asap. hopefully by > tomorrow night(BST). > > Cheers > > Santi > > > On 2020-06-09 19:20, Kashyap Desai wrote: >>> -----Original Message----- >>> From: Santiago Martinez [mailto:sm@codenetworks.net] >>> Sent: Tuesday, June 9, 2020 11:27 PM >>> To: Kashyap Desai ; Don Lewis >>> ; Andriy Gapon >>> Cc: FreeBSD Current ; Kashyap D. Desai >>> ; Kenneth D. Merry ; Sumit Saxena >>> ; Chandrakanth Patil >>> >>> Subject: Re: MRSAS Panic during Install. >>> >>> Hi! so it works but i got a lock order reversal warning, but it >>> continue. >> OK. So what is a warning ? >> >>> Wil check go it goes.... >> Could not get your point. Can you elaborate ? >> >> >> Also can you try Raid - 1 VD with below patch ? >> >> diff --git a/mrsas.c b/mrsas.c >> index 3d33073..60f4b4d 100755 >> --- a/mrsas.c >> +++ b/mrsas.c >> @@ -1744,11 +1744,14 @@ mrsas_complete_cmd(struct mrsas_softc *sc, >> u_int32_t >> MSIxIndex) >> data_length = >> r1_cmd->io_request->DataLength; >>                                                          sense = >> r1_cmd->sense; >>                                                  } >> + >> + mtx_lock(&sc->sim_lock); >> r1_cmd->ccb_ptr = NULL; >>                                                  if >> (r1_cmd->callout_owner) { >> >> callout_stop(&r1_cmd->cm_callout); >> r1_cmd->callout_owner >> = false; >>                                                  } >> + mtx_unlock(&sc->sim_lock); >> mrsas_release_mpt_cmd(r1_cmd); >> >> mrsas_map_mpt_cmd_status(cmd_mpt, >> cmd_mpt->ccb_ptr, status, >> extStatus, >> data_length, sense); >> >> >> >>> Santi >>> >>> On 2020-06-09 11:13, Santiago Martinez wrote: >>>> Trying right now, will let you know..... >>>> >>>> >>>> On 2020-06-09 11:07, Kashyap Desai wrote: >>>>> Hi Santi - Please try without Raid-1 VD. Most likely you will not >>>>> observe issue, but you can confirm from your end. >>>>> >>>>> Kashyap >>>>> >>>>>> -----Original Message----- >>>>>> From: Santiago Martinez [mailto:sm@codenetworks.net] >>>>>> Sent: Tuesday, June 9, 2020 2:08 PM >>>>>> To: Don Lewis ; Andriy Gapon >>> >>>>>> Cc: FreeBSD Current ; Kashyap D. Desai >>>>>> ; Kenneth D. Merry >>>>>> Subject: Re: MRSAS Panic during Install. >>>>>> >>>>>> Hi Kashayp, that's correct, the servers has two raids. A raid 1 VD0 >>>>>> with 2xSSD on it and a RAID5 VD0. >>>>>> >>>>>> Do you want me to break the raid and see i it does not trigger the >>>>>> bug? >>>>>> >>>>>> cheers >>>>>> >>>>>> Santi >>>>>> >>>>>> >>>>>> On 2020-06-09 07:51, Don Lewis wrote: >>>>>>> On  9 Jun, Andriy Gapon wrote: >>>>>>>> On 09/06/2020 03:42, Santiago Martinez wrote: >>>>>>>>> Hi Everyone, today I tested with 12.1 and it works without any >>>>>>>>> issues (at least for now). >>>>>>>>> >>>>>>>>> I will sync against current and see if it fails. >>>>>>>>> >>>>>>>>> Santiago >>>>>>>>> >>>>>>>>> On 2020-06-08 17:41, Santiago Martinez wrote: >>>>>>>>>> Hi there, tried again and now i got it with UFS also.. that make >>>>>>>>>> sense.. >>>>>> right... >>>>>>>>>> On 2020-06-08 15:20, Santiago Martinez wrote: >>>>>>>>>>> Hi Everyone, >>>>>>>>>>> >>>>>>>>>>> I'm installing FreeBSD current(361567) snapshot on a Lenovo >>>>>>>>>>> SR655 >>>>>> server. >>>>>>>>>>> After selecting ZFS, and the installer tries to make the >>>>>>>>>>> partitions, etc I get the following panic. >>>>>>>>>>> >>>>>>>>>>> I tried selecting UFS and its works. >>>>>>>>>>> >>>>>>>>>>> I uploaded a screenshot as I only have KVM access to it: >>>>>>>>>>> >>>>>>>>>>> https://0bin.net/paste/4yn33GkSKiYto6m4#h78yCE6h80- >>>>>> 3DsApbXa1XLW9+b >>>>>>>>>>> hoKhOr3MVS+NRgA5A >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The server is a ThinkSystem SR655, with the following >>>>>>>>>>> controller, RAID 930-8i 2GB Flash PCIe 12Gb Adapter >>>>>>>> Lousy OCR of the picture: >>>>>>>> ... >>>>>>>> nic: nutex mrsas_sin_lock not ouned at >>>>>>>> /usr/src/sys/kern/kern_nutex.c:284 >>>>>>>> ... >>>>>>>> b_trace_self_urapper () at db_trace_self_urapper+8x2b/frane >>>>>>>> BxfffffeB33c44a918 >>>>>>>> anic() at vpanic+Bx182/frane BxfffffeA33c44ad68 >>>>>>>> nic() at panic+Bx43/frame BxfffffeB33c44adcd >>>>>>>> _mtx_assert() at __mtx_assert+@xb@/frane Bxfffffed33c44a9dd >>>>>>>> callout_stop_safe() at _callout_stop_safe+Bx82/frane >>>>>>>> Bxfffffe33c44aac >>>>>>>> rsas_conplete_cnd() at mrsas_complete_cnd+8x1b8/frane >>>>>>>> BxfffffeB33c4daaed >>>>>>>> ithread_loop() at ithread_loop+@x279/frame BxfffffeB33c44ah78 >>>>>>>> >>>>>>>> This looks like a fallout from r342064. >>>>>>>> cm_callout is initialized like this: >>>>>>>>      callout_init_mtx(&cmd->cm_callout, &sc->sim_lock, 0); but in >>>>>>>> mrsas_complete_cmd() it's stopped without holding the lock. >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> freebsd-current@freebsd.org mailing list >>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>>>> To unsubscribe, send any mail to "freebsd-current- >>>>>> unsubscribe@freebsd.org" >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to >>>> "freebsd-current-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org"