Date: Tue, 23 Jun 2015 18:42:03 +0300 From: Max Gurtovoy <maxg@mellanox.com> To: John-Mark Gurney <jmg@funkthat.com> Cc: <freebsd-scsi@freebsd.org>, Sagi Grimberg <sagig@mellanox.com>, Oren Duer <oren@mellanox.com>, Hans Petter Selasky <hanss@mellanox.com> Subject: Re: gmultipath HA over iscsi/iser Message-ID: <55897E4B.8060608@mellanox.com> In-Reply-To: <20150619162015.GD96349@funkthat.com> References: <557DA8C0.1020209@mellanox.com> <20150619162015.GD96349@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi All, On 6/19/2015 7:20 PM, John-Mark Gurney wrote: > Max Gurtovoy wrote this message on Sun, Jun 14, 2015 at 19:16 +0300: >> lately I was testing HA using gmultipath utility over iSCSI/iSER devices. >> I'm working on 11-current code base. >> I created 1 LUN on the target side and connected via 2 different >> physical ports from the initiator side. >> On the initiator side I see see /dev/da0 and /dev/da1. >> I created multipath device using: >> gmultipath label dm0 /dev/da0 /dev/da1. >> Now I have new device /dev/multipath/dm0. >> I set kern.iscsi.fail_on_disconnection=1 (to fail IO fast). >> >> Issue 1: >> ------------- >> I can't run simple fio/dd traffice over /dev/da0 nor /dev/da1. >> The only traffic that possible is using the multipath device dm0. >> Is this by design ? > > This is probably because geom only lets one writer open the device at > a time, and when the dm0 device is open, it blocks any other writers > to it's underlying devices... There is a special geom debug flag that > you can set to avoid this... > I tried this debug exclusive flag with iSCSI/iSER but when I set it to 0 the system crushed at the 'gmultipath label ...' stage. John, did you try it over iSCSI ? >> In the linux implementation we can run traffic on both block devices and >> multipath devices. >> >> Issue 2: >> -------------- >> I run some fio traffic utility over multipath device dm0 on initiator >> side with port toggling in a loop >> >> Port 1 down --> sleep 2 mins (iSCSI/ISER device reconnecting meanwhile >> with no success) --> port 1 up --> sleep 5 mins (iSCSI/ISER device >> reconnecting successecfully) >> Port 2 down --> sleep 2 mins (iSCSI/ISER device reconnecting meanwhile >> with no success) --> port 2 up --> sleep 5 mins (iSCSI/ISER device >> reconnecting successecfully) >> >> The expected result is that when the port N is down than the traffic >> moves to the available port and continue succesfully. >> I run this test for many hours and traffic FAILED (even though there was >> at least 1 suitable path between initiator and target). > > Someone else will need to answer this one... > I'm pretty new in the FreeBSD community so I don't know who is the maintainer of the gmultipath driver. Maybe we can discuss about testing the gmultipath driver over iscsi/iser devices and fix some bugs together ? We are planning to add it to our test plan and HA is in high priority for us. Thanks, Max
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55897E4B.8060608>