Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 8 Nov 2012 00:35:22 -0000
From:      "Steven Hartland" <killing@multiplay.co.uk>
To:        "Doug Ambrisko" <ambrisko@ambrisko.com>
Cc:        freebsd-scsi@freebsd.org, freebsd-stable@freebsd.org
Subject:   Re: mfi panic on recused on non-recusive mutex MFI I/O lock
Message-ID:  <0B4E8AFF9DA04C6EBD2496A8B58F1D67@multiplay.co.uk>
References:  <2DC1C56CFFF24FE0B17C34AD21A7DFAA@multiplay.co.uk> <39D16C43C8274CE9B8F23C18459E2FD4@multiplay.co.uk> <20121105212911.GA17904@ambrisko.com> <27169C7FE704495087A093752D15E7B6@multiplay.co.uk> <20121106180152.GA40422@ambrisko.com> <6B5B65F4FC854EB8BBC701500096602E@multiplay.co.uk>

next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.

------=_NextPart_000_0112_01CDBD48.EFF94C40
Content-Type: text/plain;
	format=flowed;
	charset="iso-8859-1";
	reply-type=response
Content-Transfer-Encoding: 7bit

----- Original Message ----- 
From: "Steven Hartland"
>> On Tue, Nov 06, 2012 at 12:09:42AM -0000, Steven Hartland wrote:
>> | Thanks Doug, actually just finished another test run with some more
>> | debugging in and I believe I've found the reason for the non-recusive
>> | lock and at least some of the queuing issues.
>> | 
>> | The non-recursive lock is due to the mfi_tbolt_reset calling
>> | mfi_process_fw_state_chg_isr with mfi_io_lock held which in turn calls
>> | mfi_tbolt_init_MFI_queue which tries to acquire mfi_io_lock hence
>> | the problem.
>> | 
>> | mfi-lock.txt attached I believe fixes this as well as what appears
>> | to be an invalid call to mtx_unlock(&sc->mfi_io_lock) in mfi_attach
>> | which never acquires the lock as far as can see, possibly a cut and
>> | paste error.
>> 
>> I don't seem to see the attachment.
> 
> Yer seems like some mail fail by me there, but I've had some more locking
> panics during todays tests anyway, requiring additional fixes. Will update
> and post when I'm happy with it.

OK two patches attached
== zz-mfi-lock.patch ==
Fixes mfi panic on recused on non-recusive mutex MFI I/O lock

Removes a mtx_unlock call for mfi_io_lock which is never aquired

== zz-mfi-queue.patch ==
Fixes queuing issues where mfi_release_command blindly sets the cm_flags = 0
without first removing the command from the relavent queue.

This was causing panics in the queue functions which check to ensure a command
is not on another queue.

Also fixed some cases where the error from mfi_mapcmd was lost and where the
command was never released / dequeued in error cases.

Ensure that all failures to mfi_mapcmd are logged

Fixed possible null pointer exception in mfi_aen_setup if mfi_get_log_state
failed.

Fixed mfi_parse_entries & mfi_aen_setup not returning possible errors

Corrected MFI_DUMP_CMDS calls with invalid vars SC vs sc

Commands which have timed out now set cm_error to ETIMEDOUT and call
mfi_complete which prevents them getting stuck in the busy queue forever.

Fixed possible use of NULL pointer in mfi_tbolt_get_cmd

Changed output formats to be more easily recognisable when debugging.

A few style (9) fixes e.g. braced single line conditions and double blank
lines
----------

I've just had another panic, trace below, but it doesn't seem to be related
to my changes so I'd appreciate your feedback on them as they are for now.

While the lock patch fixes the problems I've seen, its not clear to me
why mfi_tbolt_reset is acquiring the lock and hence requiring
mfi_process_fw_state_chg_isr to jump through hoops to ensure locking
around queue manipulation is done correctly. Given what its doing
(resetting the entire adapter) I wouldn't be surprised if it should
really be acquiring the config lock.

Other things I've noticed / questions
* Should mfi_abort sleep even if its call to mfi_mapcmd fails?
* Should mfi_get_controller_info really ignore the error from mfi_mapcmd?
* Do these controllers not support none 512 byte requests? Currently
all syspd requests are done assuming 512 byte sectors which the disk may
not be. This will both reduce performance or potentially break totally
if the firmware isn't translating it under the surface correctly.

Anyway the new panic manually transcribed is:-
panic: Bad linx elm 0xffffff0069b0fc0 next->prev != elm
...
mfi_tbolt_get_cmd()
mfi_build_mpt_pass_thru()
mfi_tbolt_build_mpt_cmd()
mfi_tbolt_send_frame()
bus_dmamap_load()
mfi_mapcmd()
mfi_startio()
mfi_syspd_strategy()
g_disk_start()
g_io_schedule_down()
g_down_proc_body()
fork_exit()
fork_trampoline()

Looks like mfi_cmd_tbolt_tqh has become corrupt some how, but as far as I
can tell all manip is done using the TAILQ macros and under mfi_io_lock
so its not obvious to me at this time why this is, any ideas?

There was an obvious error in mfi_tbolt_get_cmd which is now fixed in the
queue patch, where cmd can be used even if queue was empty and TAILQ_FIRST
returned NULL, but I can't see this causing this panic.

This is running with a debug kernel with:-
options         WITNESS
options         INVARIANTS
options         INVARIANT_SUPPORT
options         DDB
options         GDB
options         PRINTF_BUFR_SIZE=2048
options         MFI_DEBUG

Unfortunately I've only got this hardware till Friday unfortunately so any
ideas would be most appreciated so I can get testing done before then.

    Regards
    Steve

================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
------=_NextPart_000_0112_01CDBD48.EFF94C40
Content-Type: application/octet-stream;
	name="zz-mfi-lock.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="zz-mfi-lock.patch"

Fixes mfi panic on recused on non-recusive mutex MFI I/O lock=0A=
=0A=
Removes a mtx_unlock call for mfi_io_lock which is never aquired=0A=
--- sys/dev/mfi/mfi.c.orig	2012-11-07 14:40:33.960774577 +0000=0A=
+++ sys/dev/mfi/mfi.c	2012-11-07 14:50:28.267789676 +0000=0A=
@@ -728,10 +728,8 @@=0A=
 		    "hook\n");=0A=
 		return (EINVAL);=0A=
 	}=0A=
-	if ((error =3D mfi_aen_setup(sc, 0), 0) !=3D 0) {=0A=
-		mtx_unlock(&sc->mfi_io_lock);=0A=
+	if ((error =3D mfi_aen_setup(sc, 0), 0) !=3D 0)=0A=
 		return (error);=0A=
-	}=0A=
 =0A=
 	/*=0A=
 	 * Register a shutdown handler.=0A=
--- sys/dev/mfi/mfi_tbolt.c.orig	2012-11-07 12:21:56.249116533 +0000=0A=
+++ sys/dev/mfi/mfi_tbolt.c	2012-11-07 14:50:28.268789748 +0000=0A=
@@ -1194,6 +1194,7 @@=0A=
 			sc->hw_crit_error=3D 1;=0A=
 			return ;=0A=
 		}=0A=
+		mtx_unlock(&sc->mfi_io_lock);=0A=
 		if ((error =3D mfi_tbolt_init_MFI_queue(sc)) !=3D 0)=0A=
 				return;=0A=
 =0A=
@@ -1225,7 +1226,9 @@=0A=
 			/*=0A=
 			 * Initiate AEN (Asynchronous Event Notification)=0A=
 			 */=0A=
+			mtx_unlock(&sc->mfi_io_lock);=0A=
 			mfi_aen_setup(sc, sc->last_seq_num);=0A=
+			mtx_lock(&sc->mfi_io_lock);=0A=
 			sc->issuepend_done =3D 1;=0A=
 			device_printf(sc->mfi_dev, "second stage of reset "=0A=
 			    "complete, FW is ready now.\n");=0A=
@@ -1237,7 +1240,6 @@=0A=
 		device_printf(sc->mfi_dev, "mfi_process_fw_state_chg_isr "=0A=
 		    "called with unhandled value:%d\n", sc->adpreset);=0A=
 	}=0A=
-	mtx_unlock(&sc->mfi_io_lock);=0A=
 }=0A=
 =0A=
 /*=0A=

------=_NextPart_000_0112_01CDBD48.EFF94C40
Content-Type: application/octet-stream;
	name="zz-mfi-queue.patch"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="zz-mfi-queue.patch"

Fixes queuing issues where mfi_release_command blindly sets the cm_flags =
=3D 0=0A=
without first removing the command from the relavent queue.=0A=
=0A=
This was causing panics in the queue functions which check to ensure a =
command=0A=
is not on another queue.=0A=
=0A=
Also fixed some cases where the error from mfi_mapcmd was lost and where =
the=0A=
command was never released / dequeued in error cases.=0A=
=0A=
Ensure that all failures to mfi_mapcmd are logged=0A=
=0A=
Fixed possible null pointer exception in mfi_aen_setup if =
mfi_get_log_state=0A=
failed.=0A=
=0A=
Fixed mfi_parse_entries & mfi_aen_setup not returning possible errors=0A=
=0A=
Corrected MFI_DUMP_CMDS calls with invalid vars SC vs sc=0A=
=0A=
Commands which have timed out now set cm_error to ETIMEDOUT and call=0A=
mfi_complete which prevents them getting stuck in the busy queue forever.=0A=
=0A=
Fixed possible use of NULL pointer in mfi_tbolt_get_cmd=0A=
=0A=
Changed output formats to be more easily recognisable when debugging.=0A=
=0A=
A few style (9) fixes e.g. braced single line conditions and double =
blank lines=0A=
--- sys/dev/mfi/mfi.c.orig	2012-11-07 14:50:28.267789676 +0000=0A=
+++ sys/dev/mfi/mfi.c	2012-11-07 14:55:16.768525352 +0000=0A=
@@ -837,6 +837,23 @@=0A=
 		cm->cm_sg->sg32[0].addr =3D 0;=0A=
 	}=0A=
 =0A=
+	/*=0A=
+	 * Command may be on other queues e.g. busy queue depending on the=0A=
+	 * flow of a previous call to mfi_mapcmd, so ensure its dequeued=0A=
+	 * properly=0A=
+	 */=0A=
+	if ((cm->cm_flags & MFI_ON_MFIQ_BUSY) !=3D 0)=0A=
+		mfi_remove_busy(cm);=0A=
+	if ((cm->cm_flags & MFI_ON_MFIQ_READY) !=3D 0)=0A=
+		mfi_remove_ready(cm);=0A=
+=0A=
+	/* We're not expecting it to be on any other queue but check */=0A=
+	if ((cm->cm_flags & MFI_ON_MFIQ_MASK) !=3D 0) {=0A=
+		printf("command %p is still on another queue, flags =3D %#x\n",=0A=
+		    cm, cm->cm_flags);=0A=
+		panic("command is still on a queue");=0A=
+	}=0A=
+=0A=
 	hdr_data =3D (uint32_t *)cm->cm_frame;=0A=
 	hdr_data[0] =3D 0;	/* cmd, sense_len, cmd_status, scsi_status */=0A=
 	hdr_data[1] =3D 0;	/* target_id, lun_id, cdb_len, sg_count */=0A=
@@ -950,15 +967,12 @@=0A=
 	cm->cm_data =3D NULL;=0A=
 	cm->cm_flags =3D MFI_CMD_POLLED;=0A=
 =0A=
-	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0) {=0A=
+	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0)=0A=
 		device_printf(sc->mfi_dev, "failed to send init command\n");=0A=
-		mtx_unlock(&sc->mfi_io_lock);=0A=
-		return (error);=0A=
-	}=0A=
 	mfi_release_command(cm);=0A=
 	mtx_unlock(&sc->mfi_io_lock);=0A=
 =0A=
-	return (0);=0A=
+	return (error);=0A=
 }=0A=
 =0A=
 static int=0A=
@@ -1046,27 +1060,26 @@=0A=
 	class_locale.members.evt_class  =3D mfi_event_class;=0A=
 =0A=
 	if (seq_start =3D=3D 0) {=0A=
-		error =3D mfi_get_log_state(sc, &log_state);=0A=
+		if((error =3D mfi_get_log_state(sc, &log_state)) !=3D 0)=0A=
+			goto out;=0A=
 		sc->mfi_boot_seq_num =3D log_state->boot_seq_num;=0A=
-		if (error) {=0A=
-			if (log_state)=0A=
-				free(log_state, M_MFIBUF);=0A=
-			return (error);=0A=
-		}=0A=
 =0A=
 		/*=0A=
 		 * Walk through any events that fired since the last=0A=
 		 * shutdown.=0A=
 		 */=0A=
-		mfi_parse_entries(sc, log_state->shutdown_seq_num,=0A=
-		    log_state->newest_seq_num);=0A=
+		if((error =3D mfi_parse_entries(sc, log_state->shutdown_seq_num,=0A=
+		    log_state->newest_seq_num)) !=3D 0)=0A=
+			goto out;=0A=
 		seq =3D log_state->newest_seq_num;=0A=
 	} else=0A=
 		seq =3D seq_start;=0A=
-	mfi_aen_register(sc, seq, class_locale.word);=0A=
-	free(log_state, M_MFIBUF);=0A=
+	error =3D mfi_aen_register(sc, seq, class_locale.word);=0A=
+out:=0A=
+	if (log_state)=0A=
+		free(log_state, M_MFIBUF);=0A=
 =0A=
-	return 0;=0A=
+	return (error);=0A=
 }=0A=
 =0A=
 int=0A=
@@ -1076,7 +1089,6 @@=0A=
 	mtx_assert(&sc->mfi_io_lock, MA_OWNED);=0A=
 	cm->cm_complete =3D NULL;=0A=
 =0A=
-=0A=
 	/*=0A=
 	 * MegaCli can issue a DCMD of 0.  In this case do nothing=0A=
 	 * and return 0 to it as status=0A=
@@ -1310,9 +1322,8 @@=0A=
 	cm->cm_flags =3D MFI_CMD_POLLED;=0A=
 	cm->cm_data =3D NULL;=0A=
 =0A=
-	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0) {=0A=
+	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0)=0A=
 		device_printf(sc->mfi_dev, "Failed to shutdown controller\n");=0A=
-	}=0A=
 =0A=
 	mfi_release_command(cm);=0A=
 	mtx_unlock(&sc->mfi_io_lock);=0A=
@@ -1796,6 +1807,7 @@=0A=
 			mtx_lock(&sc->mfi_io_lock);=0A=
 			mfi_release_command(cm);=0A=
 			mtx_unlock(&sc->mfi_io_lock);=0A=
+			error =3D EIO;=0A=
 			break;=0A=
 		}=0A=
 		mtx_lock(&sc->mfi_io_lock);=0A=
@@ -1824,7 +1836,7 @@=0A=
 	}=0A=
 =0A=
 	free(el, M_MFIBUF);=0A=
-	return (0);=0A=
+	return (error);=0A=
 }=0A=
 =0A=
 static int=0A=
@@ -1941,11 +1953,12 @@=0A=
 	dcmd->mbox[0]=3Did;=0A=
 	dcmd->header.scsi_status =3D 0;=0A=
 	dcmd->header.pad0 =3D 0;=0A=
-	if (mfi_mapcmd(sc, cm) !=3D 0) {=0A=
+	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0) {=0A=
 		device_printf(sc->mfi_dev,=0A=
 		    "Failed to get physical drive info %d\n", id);=0A=
 		free(pd_info, M_MFIBUF);=0A=
-		return (0);=0A=
+		mfi_release_command(cm);=0A=
+		return (error);=0A=
 	}=0A=
 	bus_dmamap_sync(sc->mfi_buffer_dmat, cm->cm_dmamap,=0A=
 	    BUS_DMASYNC_POSTREAD);=0A=
@@ -2211,11 +2224,14 @@=0A=
 	if ((hdr->cmd_status !=3D MFI_STAT_OK) || (hdr->scsi_status !=3D 0)) {=0A=
 		bio->bio_flags |=3D BIO_ERROR;=0A=
 		bio->bio_error =3D EIO;=0A=
-		device_printf(sc->mfi_dev, "I/O error, status=3D %d "=0A=
-		    "scsi_status=3D %d\n", hdr->cmd_status, hdr->scsi_status);=0A=
+		device_printf(sc->mfi_dev, "I/O error, status=3D%#x "=0A=
+		    "scsi_status=3D%#x\n", hdr->cmd_status, hdr->scsi_status);=0A=
 		mfi_print_sense(cm->cm_sc, cm->cm_sense);=0A=
 	} else if (cm->cm_error !=3D 0) {=0A=
 		bio->bio_flags |=3D BIO_ERROR;=0A=
+		bio->bio_error =3D cm->cm_error;=0A=
+		device_printf(sc->mfi_dev, "I/O error, error=3D%#x\n",=0A=
+		    cm->cm_error);=0A=
 	}=0A=
 =0A=
 	mfi_release_command(cm);=0A=
@@ -2251,6 +2267,9 @@=0A=
 =0A=
 		/* Send the command to the controller */=0A=
 		if (mfi_mapcmd(sc, cm) !=3D 0) {=0A=
+			device_printf(sc->mfi_dev, "Failed to startio\n");=0A=
+			if ((cm->cm_flags & MFI_ON_MFIQ_BUSY) !=3D 0)=0A=
+				mfi_remove_busy(cm);=0A=
 			mfi_requeue_ready(cm);=0A=
 			break;=0A=
 		}=0A=
@@ -2374,7 +2393,7 @@=0A=
 	cm->cm_extra_frames =3D (cm->cm_total_frame_size - 1) / MFI_FRAME_SIZE;=0A=
 =0A=
 	if (sc->MFA_enabled)=0A=
-			mfi_tbolt_send_frame(sc, cm);=0A=
+		mfi_tbolt_send_frame(sc, cm);=0A=
 	else=0A=
 		mfi_send_frame(sc, cm);=0A=
 =0A=
@@ -2466,7 +2485,7 @@=0A=
 {=0A=
 	struct mfi_command *cm;=0A=
 	struct mfi_abort_frame *abort;=0A=
-	int i =3D 0;=0A=
+	int i =3D 0, error;=0A=
 	uint32_t context =3D 0;=0A=
 =0A=
 	mtx_lock(&sc->mfi_io_lock);=0A=
@@ -2490,7 +2509,8 @@=0A=
 	cm->cm_data =3D NULL;=0A=
 	cm->cm_flags =3D MFI_CMD_POLLED;=0A=
 =0A=
-	mfi_mapcmd(sc, cm);=0A=
+	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0)=0A=
+		device_printf(sc->mfi_dev, "failed to abort command\n");=0A=
 	mfi_release_command(cm);=0A=
 =0A=
 	mtx_unlock(&sc->mfi_io_lock);=0A=
@@ -2506,7 +2526,7 @@=0A=
 		mtx_unlock(&sc->mfi_io_lock);=0A=
 	}=0A=
 =0A=
-	return (0);=0A=
+	return (error);=0A=
 }=0A=
 =0A=
 int=0A=
@@ -2544,7 +2564,8 @@=0A=
 	cm->cm_total_frame_size =3D MFI_IO_FRAME_SIZE;=0A=
 	cm->cm_flags =3D MFI_CMD_POLLED | MFI_CMD_DATAOUT;=0A=
 =0A=
-	error =3D mfi_mapcmd(sc, cm);=0A=
+	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0)=0A=
+		device_printf(sc->mfi_dev, "failed dump blocks\n");=0A=
 	bus_dmamap_sync(sc->mfi_buffer_dmat, cm->cm_dmamap,=0A=
 	    BUS_DMASYNC_POSTWRITE);=0A=
 	bus_dmamap_unload(sc->mfi_buffer_dmat, cm->cm_dmamap);=0A=
@@ -2587,7 +2608,8 @@=0A=
 	cm->cm_total_frame_size =3D MFI_PASS_FRAME_SIZE;=0A=
 	cm->cm_flags =3D MFI_CMD_POLLED | MFI_CMD_DATAOUT | MFI_CMD_SCSI;=0A=
 =0A=
-	error =3D mfi_mapcmd(sc, cm);=0A=
+	if ((error =3D mfi_mapcmd(sc, cm)) !=3D 0)=0A=
+		device_printf(sc->mfi_dev, "failed dump blocks\n");=0A=
 	bus_dmamap_sync(sc->mfi_buffer_dmat, cm->cm_dmamap,=0A=
 	    BUS_DMASYNC_POSTWRITE);=0A=
 	bus_dmamap_unload(sc->mfi_buffer_dmat, cm->cm_dmamap);=0A=
@@ -3643,7 +3665,7 @@=0A=
 =0A=
 #if 0=0A=
 		if (timedout)=0A=
-			MFI_DUMP_CMDS(SC);=0A=
+			MFI_DUMP_CMDS(sc);=0A=
 #endif=0A=
 =0A=
 		mtx_unlock(&sc->mfi_io_lock);=0A=
@@ -3656,7 +3678,7 @@=0A=
 mfi_timeout(void *data)=0A=
 {=0A=
 	struct mfi_softc *sc =3D (struct mfi_softc *)data;=0A=
-	struct mfi_command *cm;=0A=
+	struct mfi_command *cm, *tmp;=0A=
 	time_t deadline;=0A=
 	int timedout =3D 0;=0A=
 =0A=
@@ -3669,7 +3691,7 @@=0A=
 		}=0A=
 	}=0A=
 	mtx_lock(&sc->mfi_io_lock);=0A=
-	TAILQ_FOREACH(cm, &sc->mfi_busy, cm_link) {=0A=
+	TAILQ_FOREACH_SAFE(cm, &sc->mfi_busy, cm_link, tmp) {=0A=
 		if (sc->mfi_aen_cm =3D=3D cm || sc->mfi_map_sync_cm =3D=3D cm)=0A=
 			continue;=0A=
 		if (cm->cm_timestamp < deadline) {=0A=
@@ -3682,6 +3704,13 @@=0A=
 				     );=0A=
 				MFI_PRINT_CMD(cm);=0A=
 				MFI_VALIDATE_CMD(sc, cm);=0A=
+				/*=0A=
+				 * Fail the command instead of leaving it on=0A=
+				 * the queue where it could remain stuck forever=0A=
+				 */=0A=
+				mfi_remove_busy(cm);=0A=
+				cm->cm_error =3D ETIMEDOUT;=0A=
+				mfi_complete(sc, cm);=0A=
 				timedout++;=0A=
 			}=0A=
 		}=0A=
@@ -3689,7 +3718,7 @@=0A=
 =0A=
 #if 0=0A=
 	if (timedout)=0A=
-		MFI_DUMP_CMDS(SC);=0A=
+		MFI_DUMP_CMDS(sc);=0A=
 #endif=0A=
 =0A=
 	mtx_unlock(&sc->mfi_io_lock);=0A=
--- sys/dev/mfi/mfi_tbolt.c.orig	2012-11-07 23:00:24.542124476 +0000=0A=
+++ sys/dev/mfi/mfi_tbolt.c	2012-11-07 23:01:46.848207655 +0000=0A=
@@ -162,14 +162,14 @@=0A=
 	while (!( HostDiag & DIAG_WRITE_ENABLE)) {=0A=
 		for (i =3D 0; i < 1000; i++);=0A=
 		HostDiag =3D (uint32_t)MFI_READ4(sc, MFI_HDR);=0A=
-		device_printf(sc->mfi_dev, "ADP_RESET_TBOLT: retry time=3D%x, "=0A=
-		    "hostdiag=3D%x\n", retry, HostDiag);=0A=
+		device_printf(sc->mfi_dev, "ADP_RESET_TBOLT: retry time=3D%d, "=0A=
+		    "hostdiag=3D%#x\n", retry, HostDiag);=0A=
 =0A=
 		if (retry++ >=3D 100)=0A=
 			return 1;=0A=
 	}=0A=
 =0A=
-	device_printf(sc->mfi_dev, "ADP_RESET_TBOLT: HostDiag=3D%x\n", =
HostDiag);=0A=
+	device_printf(sc->mfi_dev, "ADP_RESET_TBOLT: HostDiag=3D%#x\n", =
HostDiag);=0A=
 =0A=
 	MFI_WRITE4(sc, MFI_HDR, (HostDiag | DIAG_RESET_ADAPTER));=0A=
 =0A=
@@ -181,8 +181,8 @@=0A=
 	while (HostDiag & DIAG_RESET_ADAPTER) {=0A=
 		for (i =3D 0; i < 1000; i++) ;=0A=
 		HostDiag =3D (uint32_t)MFI_READ4(sc, MFI_RSR);=0A=
-		device_printf(sc->mfi_dev, "ADP_RESET_TBOLT: retry time=3D%x, "=0A=
-		    "hostdiag=3D%x\n", retry, HostDiag);=0A=
+		device_printf(sc->mfi_dev, "ADP_RESET_TBOLT: retry time=3D%d, "=0A=
+		    "hostdiag=3D%#x\n", retry, HostDiag);=0A=
 =0A=
 		if (retry++ >=3D 1000)=0A=
 			return 1;=0A=
@@ -734,7 +734,8 @@=0A=
 =0A=
 	mtx_assert(&sc->mfi_io_lock, MA_OWNED);=0A=
 =0A=
-	cmd =3D TAILQ_FIRST(&sc->mfi_cmd_tbolt_tqh);=0A=
+	if ((cmd =3D TAILQ_FIRST(&sc->mfi_cmd_tbolt_tqh)) =3D=3D NULL)=0A=
+		return NULL;=0A=
 	TAILQ_REMOVE(&sc->mfi_cmd_tbolt_tqh, cmd, next);=0A=
 	memset((uint8_t *)cmd->sg_frame, 0, MEGASAS_MAX_SZ_CHAIN_FRAME);=0A=
 	memset((uint8_t *)cmd->io_request, 0,=0A=
@@ -1119,9 +1120,9 @@=0A=
 		 * should be performed on the controller=0A=
 		 */=0A=
 		if (cm->retry_for_fw_reset =3D=3D 3) {=0A=
-			device_printf(sc->mfi_dev, "megaraid_sas: command %d "=0A=
-			    "was tried multiple times during adapter reset"=0A=
-			    "Shutting down the HBA\n", cm->cm_index);=0A=
+			device_printf(sc->mfi_dev, "megaraid_sas: command %p "=0A=
+			    "index=3D%d was tried multiple times during adapter "=0A=
+			    "reset - Shutting down the HBA\n", cm, cm->cm_index);=0A=
 			mfi_kill_hba(sc);=0A=
 			sc->hw_crit_error =3D 1;=0A=
 			return;=0A=
@@ -1137,8 +1138,8 @@=0A=
 				if (cm->cm_frame->dcmd.opcode !=3D=0A=
 				    MFI_DCMD_CTRL_EVENT_WAIT) {=0A=
 					device_printf(sc->mfi_dev,=0A=
-					    "APJ ****requeue command %d \n",=0A=
-					    cm->cm_index);=0A=
+					    "APJ ****requeue command %p "=0A=
+					    "index=3D%d\n", cm, cm->cm_index);=0A=
 					mfi_requeue_ready(cm);=0A=
 				}=0A=
 			}=0A=
@@ -1357,6 +1358,8 @@=0A=
 		device_printf(sc->mfi_dev, "failed to send map sync\n");=0A=
 		free(ld_sync, M_MFIBUF);=0A=
 		sc->mfi_map_sync_cm =3D NULL;=0A=
+		if ((cmd->cm_flags & MFI_ON_MFIQ_BUSY) !=3D 0)=0A=
+			mfi_remove_busy(cmd);=0A=
 		mfi_requeue_ready(cmd);=0A=
 		goto out;=0A=
 	}=0A=

------=_NextPart_000_0112_01CDBD48.EFF94C40--




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0B4E8AFF9DA04C6EBD2496A8B58F1D67>