From owner-freebsd-questions@freebsd.org  Sat Sep  2 14:23:00 2017
Return-Path: <owner-freebsd-questions@freebsd.org>
Delivered-To: freebsd-questions@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1E1EE1591B
 for <freebsd-questions@mailman.ysv.freebsd.org>;
 Sat,  2 Sep 2017 14:23:00 +0000 (UTC)
 (envelope-from frank2@fjl.co.uk)
Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 56BA863C95
 for <freebsd-questions@freebsd.org>; Sat,  2 Sep 2017 14:22:59 +0000 (UTC)
 (envelope-from frank2@fjl.co.uk)
Received: from [192.168.1.35] (host86-191-18-76.range86-191.btcentralplus.com
 [86.191.18.76]) (authenticated bits=0)
 by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id v82EMob8087804
 for <freebsd-questions@freebsd.org>; Sat, 2 Sep 2017 15:22:50 +0100 (BST)
 (envelope-from frank2@fjl.co.uk)
Subject: Re: Fun with HAST and inter-host connections
To: freebsd-questions@freebsd.org
References: <6cc75798-b7f2-b794-faec-8807616fd7f4@fjl.co.uk>
 <4199b174-63df-a6a8-6e8e-0e7330dd189f@laverenz.de>
 <59A8201D.5010702@fjl.co.uk>
 <26f53e78-e9e2-d424-fc1d-b0d14b775522@laverenz.de>
From: Frank Leonhardt <frank2@fjl.co.uk>
Message-ID: <3c89400f-7964-fb0d-6778-84fef964a404@fjl.co.uk>
Date: Sat, 2 Sep 2017 15:22:52 +0100
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <26f53e78-e9e2-d424-fc1d-b0d14b775522@laverenz.de>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.23
X-BeenThere: freebsd-questions@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: User questions <freebsd-questions.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-questions>, 
 <mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions/>
List-Post: <mailto:freebsd-questions@freebsd.org>
List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
 <mailto:freebsd-questions-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 02 Sep 2017 14:23:00 -0000

On 01/09/2017 09:58, Uwe Laverenz wrote:
> Hi,
>
> Am 31.08.2017 um 16:41 schrieb Frank Leonhardt:
>
>> The problem with link aggregation is that you need lots of 1Gb 
>> Ethernet cards and run out PCI slots pretty quickly. Or are you 
>> suggesting 
>
> Not necessarily, you could use PCIe cards with 2 or 4 ports.
>
>> multiple targets bound to a particular interface? Not a bad idea, but 
>
> Yes, multiple networks, one target per network, round robin policy.
>
>> The PCIe is also going to be a bottleneck, but I'm ignoring this for 
>> now.
>
> Are you sure? PCIe 3.0 supports 985MB/s on x1.
>
>> What I really want is the fastest way to connect to boxes using IP 
>> (preferably). 
>
> Then 10Gb/s ethernet would probably be the best option IMHO. 

That's what I thought, but 10G is expensive and no fun. I've got my PCIe 
slots full of HBAs and no space for multiple Ethernet, and it'd take 10+ 
x 1G ports to match 1x10G. I've experimented with 2-port cards; 4-port 
cards are usually a bit pricey (apart from some Realtek ones).

I wonder if you can still get duel-ported RAM? (And I did think of using 
nearline drives, before anyone suggests that).

The $30 USB 3.1 cards look the most fun right now. (And yes, PCI is a 
bottleneck because I'm using old motherboards for experimentation). :-(

Regards, Frank.