From owner-freebsd-hackers@FreeBSD.ORG  Tue May 19 21:22:12 2009
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 605A91065675
	for <hackers@freebsd.org>; Tue, 19 May 2009 21:22:12 +0000 (UTC)
	(envelope-from steve@Watt.COM)
Received: from wattres.watt.com (wattres.watt.com [66.93.133.130])
	by mx1.freebsd.org (Postfix) with ESMTP id 403F48FC0C
	for <hackers@freebsd.org>; Tue, 19 May 2009 21:22:11 +0000 (UTC)
	(envelope-from steve@Watt.COM)
Received: from wattres.watt.com (localhost.watt.com [127.0.0.1])
	by wattres.watt.com (8.14.3/8.14.3) with ESMTP id n4JKwSfY031383
	for <hackers@freebsd.org>; Tue, 19 May 2009 13:58:28 -0700 (PDT)
	(envelope-from steve@wattres.watt.com)
Received: (from steve@localhost)
	by wattres.watt.com (8.14.3/8.14.3/Submit) id n4JKwSAH031382
	for hackers@freebsd.org; Tue, 19 May 2009 13:58:28 -0700 (PDT)
	(envelope-from steve)
Message-Id: <200905192058.n4JKwSAH031382@wattres.watt.com>
X-Newsgroups: local.freebsd-hackers
In-Reply-To: <20090519193727.7512E4FC814@xroff.net>
From: steve@Watt.COM (Steve Watt)
Organization: Watt Consultants, San Jose, CA, USA
Date: Tue, 19 May 2009 13:58:28 -0700
X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98)
To: hackers@freebsd.org
X-Archived: 1242766708.073375022@wattres.Watt.COM
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1
	(wattres.watt.com [127.0.0.1]);
	Tue, 19 May 2009 13:58:28 -0700 (PDT)
Cc: 
Subject: Re: Question about PCIe networks
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, 
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 May 2009 21:22:12 -0000

In <20090519193727.7512E4FC814@xroff.net>, emorras@xroff.net write:
>I have read recently this
>http://www.wwpi.com/hardware/hardware/6540-ethernet-tunneling-through-pci-express-inter-processor-communication-low-latency-storage-io

(From a company that makes PCIe switches to connect multiple root
complexes together.)

>It's about using PCIe to connect 2 servers directly, without using 
>ethernet or other hardware.
>
>Can it be done in FreeBSD? What is needed to know?

Certainly.  Non-transparent PCIe bridges basically create windows
of memory space into the other side.  You'd need the two sides to
agree on the data structures, and what signalling mechanism to
use for packet availability.  Quite straightforward, really.

What you'll need is the bridge hardware that connects to the two
systems, the two systems, datasheets, and some time.  Each side
of the bridge would allocate some DMAable memory, and set up the
bridge so that is visible to the other side.  Set up a pair
of rings (one per direction of traffic), and go.

-- 
Steve Watt KD6GGD  PP-ASEL-IA          ICBM: 121W 56' 57.5" / 37N 20' 15.3"
 Internet: steve @ Watt.COM                      Whois: SW32-ARIN
   Free time?  There's no such thing.  It just comes in varying prices...