From owner-freebsd-stable@FreeBSD.ORG Sun Mar 11 20:33:07 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43AA6106564A for ; Sun, 11 Mar 2012 20:33:07 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id BB3688FC1B for ; Sun, 11 Mar 2012 20:33:06 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so2988416bkc.13 for ; Sun, 11 Mar 2012 13:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=sJrczfdUZEmRRidJLrzqguee+cEFnxRo3AlpU4ho44I=; b=HSJUXRbW0k756zGabbZ26WFxOutS/s0eqPqpWD0eGirjoEDCbjyhHvKQbYIci7Ol9E tQdUyuGPnePoFKL3zrTlylq/f4sA28wjYu6OsUz0PXolFjPaOuwqhDoayCvyVu/xggwv YbMAqRmJXEjWBGcRLE9nLF4ZjgsMpqvP3vqGC2kSr2+AB5YUxvBOfLgx04aEvahPJ4qo zPhUUA+KTEhXuPjvjU1l+I709aCDw+ynwH6yGRBnwc5M1J4D0WZT2LNT0EMLMN01rRRW qmDsePqS5Rkd4/bxw/r68d7WMXTIoXd/IBgpnv5zd2F2G2+pOlr8HackFxV1a37maEpc qqJQ== Received: by 10.204.156.139 with SMTP id x11mr4026455bkw.59.1331497985438; Sun, 11 Mar 2012 13:33:05 -0700 (PDT) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id m3sm20960261bkz.0.2012.03.11.13.33.02 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 11 Mar 2012 13:33:03 -0700 (PDT) From: Mikolaj Golub To: Phil Regnauld References: <20120311185457.GB1684@macbook.bluepipe.net> X-Comment-To: Phil Regnauld Sender: Mikolaj Golub Date: Sun, 11 Mar 2012 22:31:46 +0200 In-Reply-To: <20120311185457.GB1684@macbook.bluepipe.net> (Phil Regnauld's message of "Sun, 11 Mar 2012 19:54:57 +0100") Message-ID: <861uoyvpzh.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-stable@freebsd.org Subject: Re: Issue with hast replication X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Mar 2012 20:33:07 -0000 On Sun, 11 Mar 2012 19:54:57 +0100 Phil Regnauld wrote: PR> Hi, PR> I've got a fairly simple setup: two hosts running 9.0-R (will upgrade to stable PR> if told to, but want to check here first), ZFS and HAST. HAST is configured to PR> run on top of zvols configured on each host, as illustrated: PR> FS FS PR> +------+ +------+ PR> | hvol | <---- hastd -----> | hvol | PR> +------+ +------+ PR> | zvol | | zvol | PR> +------+ +------+ PR> | zfs | | zfs | PR> +------+ +------+ PR> h1 h2 PR> Connection is gigabit to the same switch. No issues with large TCP PR> transfers such as SCP/FTP. PR> Config is vanilla: PR> # zfs create -V 10G zfs/hvol PR> hast.conf: PR> resource hvol { PR> on h1 { PR> local /dev/zvol/zfs/hvol PR> remote tcp4://192.168.1.100 PR> } PR> on h2 { PR> local /dev/zvol/zfs/hvol PR> remote tcp4://192.168.1.200 PR> } PR> } PR> h1 is behaving fine as primary, either with h2 turned off or in init - PR> but as soon as I set the role to secondary for h2, the receiver PR> repeatedly crashes and restarts - see the traces below. PR> Primary: PR> Mar 11 02:02:30 h1 hastd[2282]: [hvol] (primary) Disconnected from tcp4://192.168.1.200. PR> Mar 11 02:02:30 h1 hastd[2282]: [hvol] (primary) Unable to write synchronization data: Cannot allocate memory. PR> Mar 11 02:02:41 h1 hastd[2282]: [hvol] (primary) Unable to send request (Cannot allocate memory): WRITE(31642091520, 131072). 31642091520 looks like rather large offset for 10Gb volume... Just to be more confident that this is a HAST issue could you please try the following experiment? 1) Stop hastd on h2. 2) On h1 run something like below: dd if=/dev/zvol/zfs/hvol bs=131072 | ssh h2 dd bs=131072 of=/dev/zvol/zfs/hvol (copy hvol from h1 to h2 without hastd to see if it will succeed). Note: you will need to recreate HAST provider on secondary after this. -- Mikolaj Golub