From owner-freebsd-fs@freebsd.org Thu Oct 29 09:17:42 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C853A1E17E for ; Thu, 29 Oct 2015 09:17:42 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E8DD21F7D for ; Thu, 29 Oct 2015 09:17:41 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wmeg8 with SMTP id g8so20647425wme.0 for ; Thu, 29 Oct 2015 02:17:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay_co_uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=7yeObzkoazgeuz58B9Heww4m4Ec7+Nsx+OGn7L5aK5k=; b=P5bIF9uAzeB8+xBfnY8Pl1GB3kMVKOyZ5POhfoUkj57arxDCwcV/w028POeyZXVxxC txj1YuDcPTu1Xp3Hdh0llxbCefBo8MCrgbyTC54zjyMknhf9YTmSe3eU60nhIuAaU8nJ 6N7OmJyNRocdkRzRBcncs3Bdej9VEQlAHiUntUBtKFIf3utDepw7ZgDNND9/KC4VNYd4 ZNkgdv/92/AvwAarwpOWnrA5Y+BGoxjfYtuX3ek596aTVY7EoM5TepCGURZm4HQDCuKo fxHzj0gZ1/fzekuyDbpLeguh7fNueewCRfsLQCJrwaXUKKSLOBMhdkQkqP69d3LUq191 T5Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=7yeObzkoazgeuz58B9Heww4m4Ec7+Nsx+OGn7L5aK5k=; b=dD0GNCe2u16tcVCBK+8sPuEAIjBsgxFoOOE3FarS7Qx4P9dxchgKYYn5jIdmILqRCv zSCtWz4jSuXFkETijzv67MDKgHiFPupyvCcDe3cAGG7gxGT34jZX1Zs1IvUSkNjjwwFD sZMn3Et/QZzbSyHkfEGII8qrWEjWHZI72rjOfc0w++VrYbEJKotzT5vn4aFG/LFZPy6A tH8pHBXSMJaDVtD6JDTC9IA3Iw8kcDZGbL83cNMrdCl6C5V4sQI9IlfHgRFtj0nCaVrd 1WQ1dV0rr6pyFHjZ8+eOGdDihbB+gNBp3Rt2G1dxxwiFjX2fuNtxF2W0DLu2jUTXKHRn 3irw== X-Gm-Message-State: ALoCoQnPVWyCtH9oxmxeRcXVjtjzOvww+P4VkxKoAWMsIsb+1Llyz98YbDoq4fBAR+gw7p0qvcY0 X-Received: by 10.28.135.206 with SMTP id j197mr2541981wmd.14.1446110259971; Thu, 29 Oct 2015 02:17:39 -0700 (PDT) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id cr5sm746873wjb.16.2015.10.29.02.17.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 29 Oct 2015 02:17:38 -0700 (PDT) Subject: Re: iSCSI/ZFS strangeness To: freebsd-fs@freebsd.org References: <20151029015721.GA95057@mail.michaelwlucas.com> From: Steven Hartland Message-ID: <5631E43A.4050201@multiplay.co.uk> Date: Thu, 29 Oct 2015 09:17:46 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 09:17:42 -0000 On 29/10/2015 06:42, Ben RUBSON wrote: >> Le 29 oct. 2015 à 03:26, Philip Murray a écrit : >> >> >>> On 29/10/2015, at 2:57 PM, Michael W. Lucas wrote: >>> >>> Hi, >>> >>> I'm experimenting with iSCSI HA with FreeBSD 10.2 amd64. I know people >>> do this sort of thing, but I can't figure out what I'm missing. (Most >>> of the tutorials cover HAST instead). I suspect the real problem is >>> "Lucas doesn't know the right search terms." >>> >>> The goal is to make an iSCSI-based ZFS pool that's available to two >>> separate hosts, and remains available even if one of the iSCSI servers >>> fails. Instead, the pool hangs when either of the iSCSI servers goes >>> down. >> I’m no expert and have never used iSCSI and FreeBSD before, but I think you might >> want to look at the kern.iscsi.fail_on_disconnection sysctl. >> >> man 4 iscsi >> >> That means the devices will fail instead of hang, and ZFS might decide to mark those >> devices as faulted instead of waiting for them to respond. > Hi, > > Yes, kern.iscsi.fail_on_disconnection=1 in sysctl.conf will do the trick. > In addition, you can tune the following to minimize the impact / freeze when iSCSI fails : > kern.iscsi.ping_timeout=10 > kern.iscsi.iscsid_timeout=10 > kern.iscsi.login_timeout=10 > > In addition, in ctl.conf, perhaps you should use one target instruction per disk. > I began with all my disks/luns into a unique target configuration, but when a disk fails and you replace it, you need to refresh the iSCSI configuration using iscsictl -M. At this moment you will loose the connection to all disks, leading into many ZFS devices failed... > So I think a target per disks is better, then you only refresh the dead target : iscsictl -M -i > > And last thing, thank you very much for your quality books, a must-have :) > While not strictly a pure ZFS solution, if your initiators can't both be active at the same time a better solution might be to use geom multipath with ZFS on top of that? Regards Steve