From owner-freebsd-scsi@FreeBSD.ORG  Fri Aug 24 02:08:31 2012
Return-Path: <owner-freebsd-scsi@FreeBSD.ORG>
Delivered-To: freebsd-scsi@FreeBSD.org
Received: by hub.freebsd.org (Postfix, from userid 821)
	id D87A5106566B; Fri, 24 Aug 2012 02:08:31 +0000 (UTC)
Date: Fri, 24 Aug 2012 02:08:31 +0000
From: John <jwd@FreeBSD.org>
To: FreeBSD SCSI <freebsd-scsi@FreeBSD.org>
Message-ID: <20120824020831.GA72562@FreeBSD.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.1i
Cc: 
Subject: 9.1-PRERELEASE Performance slowdown with gmultipath Active/Active
X-BeenThere: freebsd-scsi@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: SCSI subsystem <freebsd-scsi.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-scsi>,
	<mailto:freebsd-scsi-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-scsi>
List-Post: <mailto:freebsd-scsi@freebsd.org>
List-Help: <mailto:freebsd-scsi-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-scsi>,
	<mailto:freebsd-scsi-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Aug 2012 02:08:31 -0000

Hi Folks,

   A colleague was setting up a system and noticed some performance
differences. After poking around for a while, we found that gmultipath
in active/active mode is slower than active/passive:

#gmultipath configure -A Z181
#gmultipath status Z181
          Name   Status  Components
multipath/Z181  OPTIMAL  da180 (ACTIVE)
                         da372 (ACTIVE)
#dd if=/dev/da180 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.690743 secs (151804079 bytes/sec)
#dd if=/dev/da372 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681540 secs (153853916 bytes/sec)
#dd if=/dev/multipath/Z181 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 1.531437 secs (68470064 bytes/sec)

  Above, note the multipath device is 2x slower.

#gmultipath configure -P Z181
#gmultipath status Z181
          Name   Status  Components
multipath/Z181  OPTIMAL  da180 (ACTIVE)
                         da372 (PASSIVE)
#dd if=/dev/da180 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.691087 secs (151728507 bytes/sec)
#dd if=/dev/da372 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681589 secs (153842830 bytes/sec)
#dd if=/dev/multipath/Z181 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681605 secs (153839278 bytes/sec)

   In A/P mode, the performance is equal.


#gmultipath rotate Z181
#gmultipath status Z181
          Name   Status  Components
multipath/Z181  OPTIMAL  da180 (PASSIVE)
                         da372 (ACTIVE)
#dd if=/dev/da180 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.685385 secs (152990803 bytes/sec)
#dd if=/dev/da372 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681613 secs (153837395 bytes/sec)
#dd if=/dev/multipath/Z181 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681614 secs (153837233 bytes/sec)

Using the other path the performance is also the same.


The drives:

# camcontrol inquiry da180
pass190: <HP EG0600FBDSR HPD4> Fixed Direct Access SCSI-5 device 
pass190: Serial Number EA01PB70SKG41128
pass190: 600.000MB/s transfers, Command Queueing Enabled
# camcontrol inquiry da372
pass390: <HP EG0600FBDSR HPD4> Fixed Direct Access SCSI-5 device 
pass390: Serial Number EA01PB70SKG41128
pass390: 600.000MB/s transfers, Command Queueing Enabled


A complete dmesg from the system - a 9.1-prerelease

FreeBSD 9.1-PRERELEASE #0 r239482M: Tue Aug 21 22:49:16 EDT 2012


here: http://people.freebsd.org/~jwd/dmesg.boot.txt



Any thoughts on what this might be or the best way to debug it?

Thanks,
John