From owner-freebsd-drivers@freebsd.org Wed Feb 19 08:22:32 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 52D2A252C41; Wed, 19 Feb 2020 08:22:32 +0000 (UTC) (envelope-from babupalit@gmail.com) Received: from mail-oi1-x229.google.com (mail-oi1-x229.google.com [IPv6:2607:f8b0:4864:20::229]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48MrLZ5qXYz4RNw; Wed, 19 Feb 2020 08:22:30 +0000 (UTC) (envelope-from babupalit@gmail.com) Received: by mail-oi1-x229.google.com with SMTP id q84so23017402oic.4; Wed, 19 Feb 2020 00:22:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=4J3GeLY3nbFQj1QR48LzVMbrsfkFteeIrBeO23XvwXg=; b=LXlc6E32KIvten3dGRz5a+ohXHhrCj6/JEQSr6Jcr96WAnPVehMBKiJ3P4DERVzKIG QZ8fkJuIo87mS3FAcunux+uscC5W2cejp2zGUzrk2CrkJ1H0a6nV9LTL2R17M7v1Hl6y 4DGfJdaZFxFEn/19rCdgC4HGMrOl+X5iA2XejMI7rqvte6Mh9R4Evb32+81Cks1uDDhN 1HgsWNdXIIhVTToXyyF2N4R9J8q4QKIHOMBCgSZOC5NyAFqhelkno+H1Xs+j5/eXxwiV eKrqenu9mE4zr/Km5jzCD68E/hCjAchCImcdIjtAdENW93m+ouNsJftVHEWt/qLebbMv Rm1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4J3GeLY3nbFQj1QR48LzVMbrsfkFteeIrBeO23XvwXg=; b=n6tILMGoKCXIgfl1lMt13dn5qh2mDegD3ovhWY0x9Uu+Pl2eJ/iIY8zExIIeyBb2VL 3TsxqZrw2/csA85BR0huJmnpeSqV/My7kNbuJ0XLemKgirZiZNjdG/84I4GU0clmkynC GW5dEGoV3+HWjBn98wzaaaskgXVLK1lAzyLfCsYnuir7rxVNGl2vOm2MoG1mr7/dkc3r QMpm6HQ/GGTh/ptvzPNtb8MvLDBjalXRZ9B9Sbo/Pm9yHylnWT+UnR5xA1JqWMd1Q9WA I9UKgh5+u19G+PM0Vn5IfpD2jCIdgtiLF5urvY33XAHpklUtnDWsfeMduXZkDNSI9RP7 RmCw== X-Gm-Message-State: APjAAAXJdMZyM+oa1pqKjc7I8gmDHD0sCVwoXLGBi/1os4kqfOqqw4R8 XIaYe3GjdxMI2P7ekTfhc+o8UwcORXQ8mdt7Ud4ia+P5 X-Google-Smtp-Source: APXvYqxGRHkcA9SiXICrDljeNu3V26U3cqJvbxymXIK7jFYa9IJUjjRO2QimUUc7e/YOvc//zu0X1B1GGZX4IX74kxY= X-Received: by 2002:aca:814:: with SMTP id 20mr3942792oii.159.1582100549397; Wed, 19 Feb 2020 00:22:29 -0800 (PST) MIME-Version: 1.0 From: Arpan Palit Date: Wed, 19 Feb 2020 13:52:18 +0530 Message-ID: Subject: msleep_spin is failed to waken up even after wakeup routine is invoked for the same. To: freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org X-Rspamd-Queue-Id: 48MrLZ5qXYz4RNw X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=LXlc6E32; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of babupalit@gmail.com designates 2607:f8b0:4864:20::229 as permitted sender) smtp.mailfrom=babupalit@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.00)[ip: (-8.50), ipnet: 2607:f8b0::/32(-1.89), asn: 15169(-1.68), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[9.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2020 08:22:32 -0000 Hi, I am facing one issue where wakeup rountine call is unable to waken up a msleep_spin routine call with a timeout value of 10 Seconds. The real scenario is as follows: post a hardware command and sleep using msleep_spin routine till interrupt comes, After getting the interrupt waken up the sleeping process using wakeup_one/wakeup routine call. As there are more than 2048 command and 16 parallel threads are running, observed randomly *one or two of the posted command* is *timing out* for which the *interrupt has came and also wakeup routine is invoked *after getting the interrupt for the same command. Note: *The issue is not seen when number of commands are less than 2048 with timeout of 10 seconds. *The issue can be seen with less number of commands also when timeout value 1 second. Can anyone please provide me an optimized way to schedule the process or a better way to do the scheduling. Thanks, Arpan Palit From owner-freebsd-drivers@freebsd.org Sat Feb 22 16:03:22 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 79A8D243039 for ; Sat, 22 Feb 2020 16:03:22 +0000 (UTC) (envelope-from girgen@pingpong.se) Received: from mail-lj1-x243.google.com (mail-lj1-x243.google.com [IPv6:2a00:1450:4864:20::243]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48PtQw6BdPz48nH for ; Sat, 22 Feb 2020 16:03:20 +0000 (UTC) (envelope-from girgen@pingpong.se) Received: by mail-lj1-x243.google.com with SMTP id e18so5392310ljn.12 for ; Sat, 22 Feb 2020 08:03:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pingpong-se.20150623.gappssmtp.com; s=20150623; h=from:content-transfer-encoding:mime-version:subject:message-id:date :to; bh=6qe56WVx4lMqEoKSzeKsGfT9YscCa2KrGDn6ru2tUJk=; b=f6p9WMoit9rX21exHFsiWX5nwJZytAgjPJPhbsSU02FrRzpSQAkaoFLha2iqWCLGq8 5gMp6UCHVwMoWDErJ0n/UpPf9hUSU0E0+Tgmx4Cq5RI44BvLIpqpjdMpbyCzK6Fvdb3p JVFzBUNTYHhGY56WAvxVsYDYRzOvn/ZmSPUtbHya31PbuZOYXPBS2QaB38CcSwMWH4Ex qUYhB+YXCuhTBD19Afdz5ZQG9u2c6Sj8RHg9iqlC9IFdkbeeFxPv5tf7/frgI3I0h0xG itd6RghJG3dNj79Gr8puQaXfQ+idWx+jP+65yG6Vn8Tja4X7uzdFQtvxDcTe+ko6t6Lu bgOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:to; bh=6qe56WVx4lMqEoKSzeKsGfT9YscCa2KrGDn6ru2tUJk=; b=FWUKO4m5/IdtY6chFZ192kjKm7uN0nDEgT78UIuzsVVaW3C1sTfTZU9TIxPkHcMgBt jPBKBHN8LyIBYFriXWKCqd/eKAACRKpMQT6Q+oUXO4FbVsMec47StemkkrYTO6YZ3qOz tC4YYtXXgy+EYLIJN6swpv8PZC2RvrBfjWGqahTuEtGmsPutsDbwiR1Pdk3ufjESPhKA aqet94irk/Ujao+MVuClrHs7WqhhgfIP/yXaLJ5AwaNK6ezq+L/+yt8GJtgrW+j9cW6K 4wlKkgHqKkLJU8aNZe87MwClPj3soMPDuYKuCjnViVCO0NqnE9QdPYae26+tQCtABkXB EvSw== X-Gm-Message-State: APjAAAUiluTXNURjIOt9ExQm/L2y+H5j2I43Fsbt8f9R1tVp32rUmFSu 9zJP7TCLonAU+mW0KXDznlLQIrhqyF8= X-Google-Smtp-Source: APXvYqwxRK6sIvEDEghgn3puiV/wP4535aDQa8UAurSPqD/PZWOUjB2Ui7vS3rW8OEin/4jB6UpsVA== X-Received: by 2002:a2e:b4f6:: with SMTP id s22mr24069806ljm.218.1582387397302; Sat, 22 Feb 2020 08:03:17 -0800 (PST) Received: from [10.0.1.11] (c188-149-188-210.bredband.comhem.se. [188.149.188.210]) by smtp.gmail.com with ESMTPSA id m21sm3323291lfb.59.2020.02.22.08.03.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 22 Feb 2020 08:03:16 -0800 (PST) From: Palle Girgensohn Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Suspecting a problem with mfi Megaraid SAS driver in 12.1 Message-Id: <28ABA1F8-90BD-4556-B9ED-92638CA838E6@pingpong.se> Date: Sat, 22 Feb 2020 17:03:15 +0100 To: freebsd-drivers@freebsd.org X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 48PtQw6BdPz48nH X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=pingpong-se.20150623.gappssmtp.com header.s=20150623 header.b=f6p9WMoi; dmarc=none; spf=pass (mx1.freebsd.org: domain of girgen@pingpong.se designates 2a00:1450:4864:20::243 as permitted sender) smtp.mailfrom=girgen@pingpong.se X-Spamd-Result: default: False [-2.25 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[pingpong-se.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-drivers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[pingpong.se]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[pingpong-se.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[3.4.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-0.25)[ip: (2.89), ipnet: 2a00:1450::/32(-2.42), asn: 15169(-1.67), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Feb 2020 16:03:22 -0000 Hi! We updated a Fujisti server Friday morning from FreeBSD 11.2 -> 12.1 and = we are seeing problems with the storage. It as rather large, ~ 50 TB, ZFS volume. After upgrading, it complains = about chksum errors and has rebooted spontaneous twice in ~48 hours. Since it just happend after upgrading the OS, and since it happens on = several disks, I primarily suspect there is a software error. That is = because that is all that has changed. It could also be the controller of = course, and in that case triggered by the intentional reboots as part of = the upgrade, but I pursuing both options at the moment. It is a FTS PRAID CP400i controller, identified in FreeBSD to use the = mfi driver. It is configured as JBOD, no RAID at all, and *not* using = gpart. Any ideas or tips will be greatly appreciated. Palle $ grep mfi /boot/loader.conf=20 mfip_load=3D"yes" $ freebsd-version -ku 12.1-RELEASE-p2 12.1-RELEASE-p2 $ uname -iKmoprsUv FreeBSD 12.1-RELEASE-p2 FreeBSD 12.1-RELEASE-p2 GENERIC amd64 amd64 = GENERIC 1201000 1201000 $ zpool status -v pool: tank state: DEGRADED status: One or more devices is currently being resilvered. The pool = will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sat Feb 22 14:03:32 2020 184G scanned at 18,9M/s, 79,2G issued at 8,14M/s, 15,8T total 1,97G resilvered, 0,49% done, no estimated completion time config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 raidz2-0 DEGRADED 0 0 0 mfisyspd0 ONLINE 0 0 7 mfisyspd1 ONLINE 0 0 5 mfisyspd2 ONLINE 0 0 8 mfisyspd3 DEGRADED 0 0 12 too many = errors mfisyspd4 ONLINE 0 0 8 mfisyspd5 ONLINE 0 0 5 raidz2-1 DEGRADED 0 0 0 mfisyspd6 ONLINE 0 0 5 spare-1 DEGRADED 0 0 97 mfisyspd7 DEGRADED 0 0 0 too many = errors mfisyspd11 ONLINE 0 0 0 mfisyspd8 ONLINE 0 0 5 mfisyspd9 ONLINE 0 0 18 mfisyspd10 DEGRADED 0 0 13 too many = errors logs mirror-2 ONLINE 0 0 0 mfisyspd12p4 ONLINE 0 0 0 mfisyspd13p4 ONLINE 0 0 0 cache mfisyspd12p5 ONLINE 0 0 0 mfisyspd13p5 ONLINE 0 0 0 spares 11621498950123571546 INUSE was /dev/mfisyspd11 errors: No known data errors from /var/log/messages: Feb 22 14:07:41 hostname kernel: mfisyspd9: hard error cmd=3Dread = 2855317293-2855317844 Feb 22 14:07:41 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e80198, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:41 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:07:41 hostname kernel: mfisyspd6: hard error cmd=3Dread = 2855318195-2855318746 Feb 22 14:07:42 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e83fc0, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:42 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:42 hostname kernel: mfisyspd5: hard error cmd=3Dread = 6047123871-6047124388 Feb 22 14:07:42 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e802a8, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:42 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:42 hostname kernel: mfisyspd2: hard error cmd=3Dread = 6047123872-6047124389 Feb 22 14:07:43 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e80330, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:43 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:43 hostname kernel: mfisyspd8: hard error cmd=3Dread = 2906458877-2906459428 Feb 22 14:07:43 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82f48, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:43 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 0, ascq 0 Feb 22 14:07:43 hostname kernel: mfisyspd8: hard error cmd=3Dread = 2906459429-2906459980 Feb 22 14:07:43 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82420, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:43 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:07:43 hostname kernel: mfisyspd8: hard error cmd=3Dread = 2906459981-2906460532 Feb 22 14:07:43 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e83960, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:43 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:43 hostname kernel: mfisyspd0: hard error cmd=3Dread = 412266548-412267099 Feb 22 14:07:43 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e80220, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:43 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:07:43 hostname kernel: mfisyspd0: hard error cmd=3Dread = 412267100-412267651 Feb 22 14:07:49 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e815c8, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:49 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:49 hostname kernel: mfisyspd8: hard error cmd=3Dread = 2959727577-2959728128 Feb 22 14:07:49 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e81ed0, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:49 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:07:49 hostname kernel: mfisyspd1: hard error cmd=3Dread = 7610749043-7610749594 Feb 22 14:07:49 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82f48, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:49 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 0, ascq 0 Feb 22 14:07:49 hostname kernel: mfisyspd3: hard error cmd=3Dread = 4882180734-4882181285 Feb 22 14:07:51 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82b90, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:51 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:51 hostname kernel: mfisyspd9: hard error cmd=3Dread = 1041218706-1041219257 Feb 22 14:07:51 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e80c38, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:51 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:51 hostname kernel: mfisyspd9: hard error cmd=3Dread = 1041219258-1041219809 Feb 22 14:07:52 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e837c8, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:52 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:07:52 hostname kernel: mfisyspd6: hard error cmd=3Dread = 2960038937-2960039488 Feb 22 14:07:52 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e81e48, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:52 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 0, ascq 0 Feb 22 14:07:52 hostname kernel: mfisyspd6: hard error cmd=3Dread = 2960039489-2960040040 Feb 22 14:07:52 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82d28, status=3D0x3c, scsi_status=3D0 Feb 22 14:07:52 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:07:52 hostname kernel: mfisyspd6: hard error cmd=3Dread = 2960040041-2960040592 Feb 22 14:08:13 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e83960, status=3D0x3c, scsi_status=3D0 Feb 22 14:08:13 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:08:13 hostname kernel: mfisyspd7: hard error cmd=3Dread = 2906939820-2906940371 Feb 22 14:08:13 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e80908, status=3D0x3c, scsi_status=3D0 Feb 22 14:08:13 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:08:13 hostname kernel: mfisyspd1: hard error cmd=3Dread = 412681365-412681916 Feb 22 14:08:42 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82860, status=3D0x3c, scsi_status=3D0 Feb 22 14:08:42 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:08:42 hostname kernel: mfisyspd9: hard error cmd=3Dread = 2857339961-2857340492 Feb 22 14:08:47 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82ec0, status=3D0x3c, scsi_status=3D0 Feb 22 14:08:47 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 0, ascq 0 Feb 22 14:08:47 hostname kernel: mfisyspd7: hard error cmd=3Dread = 1041524253-1041524786 Feb 22 14:09:01 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e82068, status=3D0x3c, scsi_status=3D0 Feb 22 14:09:01 hostname kernel: mfi0: sense error 112, sense_key 5, asc = 32, ascq 0 Feb 22 14:09:01 hostname kernel: mfisyspd1: hard error cmd=3Dread = 7619627390-7619627941 Feb 22 14:09:13 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e838d8, status=3D0x3c, scsi_status=3D0 Feb 22 14:09:13 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 0, ascq 0 Feb 22 14:09:13 hostname kernel: mfisyspd10: hard error cmd=3Dread = 2960045475-2960046026 Feb 22 14:10:27 hostname kernel: mfi0: I/O error, = cmd=3D0xfffffe00a8e80e58, status=3D0x3c, scsi_status=3D0 Feb 22 14:10:27 hostname kernel: mfi0: sense error 114, sense_key 4, asc = 207, ascq 0 Feb 22 14:10:27 hostname kernel: mfisyspd7: hard error cmd=3Dread = 2906360948-2906361499 $ dmesg| grep mfi mfi0: port 0x6000-0x60ff mem = 0xc7b00000-0xc7b0ffff,0xc7a00000-0xc7afffff irq 26 at device 0.0 = numa-domain 0 on pci3 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23=20 mfi0: FW MaxCmds =3D 240, limiting to 128 mfi0: MaxCmd =3D 240, Drv MaxCmd =3D 128, MaxSgl =3D 70, state =3D = 0xb73c00f0 mfi0: 1923 (635692256s/0x0020/info) - Shutdown command received from = host mfip0: mfi0: 1924 (boot + 9s/0x0020/info) - Firmware initialization = started (PCI ID 005f/1000/1211/1734) mfi0: 1925 (boot + 9s/0x0020/info) - Firmware version 4.250.01-3869 mfi0: 1926 (boot + 11s/0x0020/info) - Package version 24.5.0-0031 mfi0: 1927 (boot + 11s/0x0020/info) - Board Revision =20 mfi0: 1928 (boot + 59s/0x0004/info) - Enclosure (SES) discovered on PD = 07(c Port 0 - 3/p1) mfi0: 1929 (boot + 60s/0x0004/info) - Enclosure PD 07(c Port 0 - 3/p1) = communication restored numa-domain 0 on mfi0 mfi0: 1930 (boot + 60s/0x0002/info) - Inserted: Encl PD 07 mfi1: port 0xf000-0xf0ff mem = 0xfbe00000-0xfbe0ffff,0xfbd00000-0xfbdfffff irq 50 at device 0.0 = numa-domain 1 on pci10 mfi1: Using MSI mfi1: Megaraid SAS driver Ver 4.23=20 mfi1: FW MaxCmds =3D 928, limiting to 128 mfi1: MaxCmd =3D 928, Drv MaxCmd =3D 128, MaxSgl =3D 70, state =3D = 0xb75003a0 mfi1: cmd_tbolt 0xfffff80004c37c00 has invalid sync_cmd_idx=3D128 - = skipping mfip1: numa-domain 1 on mfi1 mfisyspd0 numa-domain 0 on mfi0 mfisyspd0: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 8) mfisyspd0: SYSPD volume attached mfisyspd1 numa-domain 0 on mfi0 mfisyspd1: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 9) mfisyspd1: SYSPD volume attached mfisyspd2 numa-domain 0 on mfi0 mfisyspd2: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 10) mfisyspd2: SYSPD volume attached mfisyspd3 numa-domain 0 on mfi0 mfisyspd3: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 11) mfisyspd3: SYSPD volume attached mfisyspd4 numa-domain 0 on mfi0 mfisyspd4: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 12) mfisyspd4: SYSPD volume attached mfisyspd5 numa-domain 0 on mfi0 mfisyspd5: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 13) mfisyspd5: SYSPD volume attached mfisyspd6 numa-domain 0 on mfi0 mfisyspd6: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 14) mfisyspd6: SYSPD volume attached mfisyspd7 numa-domain 0 on mfi0 mfisyspd7: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 15) mfisyspd7: SYSPD volume attached mfisyspd8 numa-domain 0 on mfi0 mfisyspd8: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 16) mfisyspd8: SYSPD volume attached mfisyspd9 numa-domain 0 on mfi0 mfisyspd9: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 17) mfisyspd9: SYSPD volume attached mfisyspd10 numa-domain 0 on mfi0 mfisyspd10: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 18) mfisyspd10: SYSPD volume attached GEOM: mfisyspd0: the primary GPT table is corrupt or invalid. GEOM: mfisyspd0: using the secondary instead -- recovery strongly = advised. mfisyspd11 numa-domain 0 on mfi0 mfisyspd11: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 19) mfisyspd11: SYSPD volume attached mfi0: 1931 (boot + 60s/0x0002/info) - Inserted: PD 07(c Port 0 - 3/p1) = Info: enclPd=3D07, scsiType=3Dd, portMap=3D00, = sasAddr=3D500605b0000272bd,0000000000000000 mfi0: 1932 (boot + 60s/0x0002/info) - Inserted: PD 08(e0x07/s3) mfisyspd12 numa-domain 1 on mfi1 mfisyspd12: 114473MB (234441648 sectors) SYSPD volume (deviceid: 0) mfisyspd12: SYSPD volume attached mfisyspd13 numa-domain 1 on mfi1 mfisyspd13: 114473MB (234441648 sectors) SYSPD volume (deviceid: 1) mfisyspd13: SYSPD volume attached mfi0: 1933 (boot + 60s/0x0002/info) - Inserted: PD 08(e0x07/s3) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c5006308fbed,0000000000000000 mfi0: 1934 (boot + 60s/0x0002/info) - Inserted: PD 09(e0x07/s5) mfi0: 1935 (boot + 60s/0x0002/info) - Inserted: PD 09(e0x07/s5) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c5006309019d,0000000000000000 mfi0: 1936 (boot + 60s/0x0002/info) - Inserted: PD 0a(e0x07/s1) mfi0: 1937 (boot + 60s/0x0002/info) - Inserted: PD 0a(e0x07/s1) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c5006308fd8d,0000000000000000 mfi0: 1938 (boot + 60s/0x0002/info) - Inserted: PD 0b(e0x07/s0) mfi0: 1939 (boot + 60s/0x0002/info) - Inserted: PD 0b(e0x07/s0) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c50063090d45,0000000000000000 mfi0: 1940 (boot + 60s/0x0002/info) - Inserted: PD 0c(e0x07/s4) mfi0: 1941 (boot + 60s/0x0002/info) - Inserted: PD 0c(e0x07/s4) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c5006308f981,0000000000000000 mfi0: 1942 (boot + 60s/0x0002/info) - Inserted: PD 0d(e0x07/s2) mfi0: 1943 (boot + 60s/0x0002/info) - Inserted: PD 0d(e0x07/s2) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c500630910c1,0000000000000000 mfi0: 1944 (boot + 60s/0x0002/info) - Inserted: PD 0e(e0x07/s8) mfi0: 1945 (boot + 60s/0x0002/info) - Inserted: PD 0e(e0x07/s8) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D50000c0f01f7efb6,0000000000000000 mfi0: 1946 (boot + 60s/0x0002/info) - Inserted: PD 0f(e0x07/s7) mfi0: 1947 (boot + 60s/0x0002/info) - Inserted: PD 0f(e0x07/s7) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c50083c1a0a1,0000000000000000 mfi0: 1948 (boot + 60s/0x0002/info) - Inserted: PD 10(e0x07/s6) mfi0: 1949 (boot + 60s/0x0002/info) - Inserted: PD 10(e0x07/s6) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c50083c19bb9,0000000000000000 mfi0: 1950 (boot + 60s/0x0002/info) - Inserted: PD 11(e0x07/s11) mfi0: 1951 (boot + 60s/0x0002/info) - Inserted: PD 11(e0x07/s11) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D5000c50083c19e19,0000000000000000 mfi0: 1952 (boot + 60s/0x0002/info) - Inserted: PD 12(e0x07/s10) mfi0: 1953 (boot + 60s/0x0002/info) - Inserted: PD 12(e0x07/s10) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D50000c0f01e251e2,0000000000000000 mfi0: 1954 (boot + 60s/0x0002/info) - Inserted: PD 13(e0x07/s9) mfi0: 1955 (boot + 60s/0x0002/info) - Inserted: PD 13(e0x07/s9) Info: = enclPd=3D07, scsiType=3D0, portMap=3D00, = sasAddr=3D50000c0f0136664e,0000000000000000 mfi0: 1956 (635692335s/0x0020/info) - Time established as 02/22/20 = 13:12:15; (66 seconds since power on) GEOM: mfisyspd1: the primary GPT table is corrupt or invalid. GEOM: mfisyspd1: using the secondary instead -- recovery strongly = advised. mfi1: 1440 (635692257s/0x0020/info) - Shutdown command received from = host mfi1: 1441 (boot + 9s/0x0020/info) - Firmware initialization started = (PCI ID 005d/1000/1212/1734) mfi1: 1442 (boot + 9s/0x0020/info) - Firmware version 4.250.00-3868 mfi1: 1443 (boot + 10s/0x0008/info) - Battery Present mfi1: 1444 (boot + 10s/0x0020/info) - Package version 24.5.0-0031 mfi1: 1445 (boot + 10s/0x0020/info) - Board Revision =20 mfi1: 1446 (boot + 14s/0x0008/info) - Battery charge complete mfi1: 1447 (boot + 14s/0x0008/info) - Battery temperature is normal mfi1: 1448 (boot + 46s/0x0002/info) - Inserted: PD 00(e0xfc/s0) mfi1: 1449 (boot + 46s/0x0002/info) - Inserted: PD 00(e0xfc/s0) Info: = enclPd=3Dfc, scsiType=3D0, portMap=3D00, = sasAddr=3D4433221102000000,0000000000000000 mfi1: 1450 (boot + 46s/0x0002/info) - Inserted: PD 01(e0xfc/s1) mfi1: 1451 (boot + 46s/0x0002/info) - Inserted: PD 01(e0xfc/s1) Info: = enclPd=3Dfc, scsiType=3D0, portMap=3D01, = sasAddr=3D4433221103000000,0000000000000000 mfi1: 1452 (635692353s/0x0020/info) - Time established as 02/22/20 = 13:12:33; (82 seconds since power on) GEOM: mfisyspd2: the primary GPT table is corrupt or invalid. GEOM: mfisyspd2: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd3: the primary GPT table is corrupt or invalid. GEOM: mfisyspd3: using the secondary instead -- recovery strongly = advised. ses0 at mfi0 bus 0 scbus0 target 7 lun 0 pass1 at mfi0 bus 0 scbus0 target 8 lun 0 pass2 at mfi0 bus 0 scbus0 target 9 lun 0 pass3 at mfi0 bus 0 scbus0 target 10 lun 0 pass4 at mfi0 bus 0 scbus0 target 11 lun 0 pass5 at mfi0 bus 0 scbus0 target 12 lun 0 GEOM: mfisyspd4: the primary GPT table is corrupt or invalid. pass6 at mfi0 bus 0 scbus0 target 13 lun 0 pass7 at mfi0 bus 0 scbus0 target 14 lun 0 pass8 at mfi0 bus 0 scbus0 target 15 lun 0 pass9 at mfi0 bus 0 scbus0 target 16 lun 0 pass10 at mfi0 bus 0 scbus0 target 17 lun 0 pass11 at mfi0 bus 0 scbus0 target 18 lun 0 pass12 at mfi0 bus 0 scbus0 target 19 lun 0 pass13 at mfi1 bus 0 scbus11 target 0 lun 0 pass14 at mfi1 bus 0 scbus11 target 1 lun 0 GEOM: mfisyspd4: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd5: the primary GPT table is corrupt or invalid. GEOM: mfisyspd5: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd0: the primary GPT table is corrupt or invalid. GEOM: mfisyspd0: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd1: the primary GPT table is corrupt or invalid. GEOM: mfisyspd1: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd2: the primary GPT table is corrupt or invalid. GEOM: mfisyspd2: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd3: the primary GPT table is corrupt or invalid. GEOM: mfisyspd3: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd4: the primary GPT table is corrupt or invalid. GEOM: mfisyspd4: using the secondary instead -- recovery strongly = advised. GEOM: mfisyspd5: the primary GPT table is corrupt or invalid. GEOM: mfisyspd5: using the secondary instead -- recovery strongly = advised.