From owner-freebsd-infiniband@freebsd.org Fri Aug 7 18:32:04 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 963EC9B624C for ; Fri, 7 Aug 2015 18:32:04 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A82EC4F; Fri, 7 Aug 2015 18:32:04 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by pawu10 with SMTP id u10so94018164paw.1; Fri, 07 Aug 2015 11:32:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:date:message-id :cc:to:mime-version; bh=yZd8KqEi4uYvNdPPCSoSDMqy9SQF6pL26YKbEWirrG8=; b=rZJnywraDvXYBhAVEitVHj20O5HuxnUsb3lKrREc7oi5JqwpB5MdVgeG7ra+V0VdJ4 AB8vmneoiOdby7VsIouVpVq5h/Y8fbP72KIiZOjXDm/qrwg6EwFR4KOYP1qcrllRc/l9 Z/OmIFveHXT2q+2R3GrpoP+bw6KHewUr+D7qHmwl9EPCLV6TGhyedQBb4lf6dDriCZfP VIXg/i+iJGZN0M2dsm+XJZWgGAkWqclMmREM+h6UyzA1NBAgpO16vei0TkVSPPu/xwst /p4A8jua5EY4k/tX4FTxKV965v5KzqNoiv1qSeAXWE/g6NeH+ED4OaHPABmfA0NoGakX 3uPA== X-Received: by 10.68.241.166 with SMTP id wj6mr17451751pbc.114.1438972323833; Fri, 07 Aug 2015 11:32:03 -0700 (PDT) Received: from ?IPv6:2601:601:800:126d:bd9c:cbff:dc69:3b4? ([2601:601:800:126d:bd9c:cbff:dc69:3b4]) by smtp.gmail.com with ESMTPSA id pu2sm10844865pbb.92.2015.08.07.11.32.02 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 07 Aug 2015 11:32:03 -0700 (PDT) From: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Enable OFED/Infiniband support in 11.0-RELEASE by default? Date: Fri, 7 Aug 2015 11:32:00 -0700 Message-Id: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> Cc: Benno Rice , Hans Petter Selasky , Jeff Roberson , Mark Johnston To: freebsd-infiniband@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2015 18:32:04 -0000 Hi, One of the complaints from engineers at Isilon I=92ve received = in the past is that Infiniband/OFED stack support isn=92t enabled by = default in GENERIC. I would like to enable it by default in GENERIC to = improve test coverage by a general audience and ensure that bugs = introduced elsewhere (build bugs, network interface, kernel interface = bugs) aren=92t ignored by accident when running make tinderbox builds as = it=92s not built by default. I=92m considering just doing this for amd64, but I could be = convinced to do i386 and other architectures; I can=92t guarantee that = it will be usable on arm*, mips, powerpc, sparc64, etc though. Thanks! -NGie= From owner-freebsd-infiniband@freebsd.org Fri Aug 7 19:05:51 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B9E49B66FF for ; Fri, 7 Aug 2015 19:05:51 +0000 (UTC) (envelope-from jwbacon@tds.net) Received: from mail.tds.net (smtp-fo.tds.net [64.8.70.105]) by mx1.freebsd.org (Postfix) with ESMTP id EC6F2B3F for ; Fri, 7 Aug 2015 19:05:47 +0000 (UTC) (envelope-from jwbacon@tds.net) X_CMAE_Category: , , X-CNFS-Analysis: v=2.1 cv=SdTjd5hu c=1 sm=1 tr=0 a=af5anmkp17JVYryXVWXdZw==:117 a=af5anmkp17JVYryXVWXdZw==:17 a=K-v-2zaBAAAA:8 a=5Bwo4ec7AAAA:8 a=mxlJd-sRi4UA:10 a=N659UExz7-8A:10 a=6I5d2MoRAAAA:8 a=4kiURuWyZ9n9WhnNlAUA:9 a=pILNOxqGKmIA:10 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: andiYWNvbkB0ZHMubmV0 Authentication-Results: smtp02.tds.cmh.synacor.com smtp.user=jwbacon@tds.net; auth=pass (LOGIN) Received: from [129.89.38.199] ([129.89.38.199:40618] helo=outpaddling.cs.uwm.edu) by mail.tds.net (envelope-from ) (ecelerity 3.6.5.45644 r(Core:3.6.5.0)) with ESMTPSA (cipher=AES128-SHA) id 2A/5D-23201-ACCF4C55; Fri, 07 Aug 2015 14:45:30 -0400 Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? To: Garrett Cooper , freebsd-infiniband@FreeBSD.org References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> Cc: Mark Johnston , Jeff Roberson , Benno Rice From: Jason Bacon Message-ID: <55C4FCC2.9060704@tds.net> Date: Fri, 7 Aug 2015 13:45:22 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2015 19:05:51 -0000 I've admittedly been out of action on this for a while, but last time I checked, the IB network stack could be loaded as a kernel module, but a build world was still required to add IB support to userland tools. If this is no longer the case, please excuse my spam. If kernel stability is still a concern, perhaps we could leave the kernel module unloaded by default, but eliminate the need for a build world by enabling IB support by default outside the kernel? I *will* get back to testing this soon, as I'm in the midst of building a new 2,000-core HPC cluster and I have servers available to experiment with. I'm eager to see what kind of performance we can get from a FreeBSD file server with ZFS and FDR IB. Regards, Jason On 08/07/15 13:32, Garrett Cooper wrote: > Hi, > One of the complaints from engineers at Isilon I’ve received in the past is that Infiniband/OFED stack support isn’t enabled by default in GENERIC. I would like to enable it by default in GENERIC to improve test coverage by a general audience and ensure that bugs introduced elsewhere (build bugs, network interface, kernel interface bugs) aren’t ignored by accident when running make tinderbox builds as it’s not built by default. > I’m considering just doing this for amd64, but I could be convinced to do i386 and other architectures; I can’t guarantee that it will be usable on arm*, mips, powerpc, sparc64, etc though. > Thanks! > -NGie > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.org" -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Jason W. Bacon jwbacon@tds.net If a problem can be solved, there's no need to worry. If it cannot be solved, then worrying will do no good. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From owner-freebsd-infiniband@freebsd.org Fri Aug 7 19:09:17 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44B1B9B6752 for ; Fri, 7 Aug 2015 19:09:17 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05C2FB8C; Fri, 7 Aug 2015 19:09:16 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id D039B1FE023; Fri, 7 Aug 2015 21:09:06 +0200 (CEST) Message-ID: <55C502AA.4080408@selasky.org> Date: Fri, 07 Aug 2015 21:10:34 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Jason Bacon , Garrett Cooper , freebsd-infiniband@FreeBSD.org CC: Mark Johnston , Jeff Roberson , Benno Rice Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> <55C4FCC2.9060704@tds.net> In-Reply-To: <55C4FCC2.9060704@tds.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2015 19:09:17 -0000 Hi, See here: https://reviews.freebsd.org/D3182 --HPS From owner-freebsd-infiniband@freebsd.org Fri Aug 7 19:30:20 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47F979B6D5F for ; Fri, 7 Aug 2015 19:30:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qg0-x235.google.com (mail-qg0-x235.google.com [IPv6:2607:f8b0:400d:c04::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0328C1876; Fri, 7 Aug 2015 19:30:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by qged69 with SMTP id d69so81661629qge.0; Fri, 07 Aug 2015 12:30:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=vi33kcY1rMureiq2dIcnAsO2Px43hwKMA4L+eJY8JEs=; b=cCJNOyaBaANspj3VNsx94fPbEe4ptyaLruY4Ii3mMIA1esDs+awSbnA1awHts9GPj3 vSqTfWc0Ilt/xr9F1Qt+6tlunBzAVavjiWQMYu4Xszm3HJj8jPfPEasniGznEn3xy328 gbDJZ3fMBXTUoVmJwSeitWiu34E5b1K1wLKzvZorBpuHqoqIoRgORZLerzYphpAwvZON XH9rer/JriQkzvh+Iye8hX/o3CVK88830N3E+Yb2v9rlc6MSVowV8Zg2NgTBH9wV12NG 24vio1JVrT7gmJ2GIfQbg/HCxVxnEYpdEIjFnpZCbJDXjutW0cYKxJyBuhrON2HQWiwg uDDg== X-Received: by 10.140.94.194 with SMTP id g60mr16016985qge.72.1438975819055; Fri, 07 Aug 2015 12:30:19 -0700 (PDT) Received: from muskytusk ([104.236.250.12]) by smtp.gmail.com with ESMTPSA id a8sm5469539qka.0.2015.08.07.12.30.17 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Aug 2015 12:30:18 -0700 (PDT) Sender: Mark Johnston Date: Fri, 7 Aug 2015 19:29:30 +0000 From: Mark Johnston To: Garrett Cooper Cc: freebsd-infiniband@FreeBSD.org, Benno Rice , Hans Petter Selasky , Jeff Roberson Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? Message-ID: <20150807192930.GA88925@muskytusk> References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2015 19:30:20 -0000 On Fri, Aug 07, 2015 at 11:32:00AM -0700, Garrett Cooper wrote: > Hi, > One of the complaints from engineers at Isilon I’ve received in the past is that Infiniband/OFED stack support isn’t enabled by default in GENERIC. I would like to enable it by default in GENERIC to improve test coverage by a general audience and ensure that bugs introduced elsewhere (build bugs, network interface, kernel interface bugs) aren’t ignored by accident when running make tinderbox builds as it’s not built by default. make tinderbox will build LINT kernels, which for amd64 will include the OFED stack. As Jason pointed out, all of the IB stack (including the Linux compat shims) can already be built as a KLD. Why not just make WITH_OFED the default on amd64 instead? That way the KLDs and userland tools will be built by default, and the size of the kernel needn't grow. > I’m considering just doing this for amd64, but I could be convinced to do i386 and other architectures; I can’t guarantee that it will be usable on arm*, mips, powerpc, sparc64, etc though. From owner-freebsd-infiniband@freebsd.org Fri Aug 7 21:46:34 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 433529B6472 for ; Fri, 7 Aug 2015 21:46:34 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pa0-x230.google.com (mail-pa0-x230.google.com [IPv6:2607:f8b0:400e:c03::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17209F93 for ; Fri, 7 Aug 2015 21:46:34 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by pacrr5 with SMTP id rr5so60666037pac.3 for ; Fri, 07 Aug 2015 14:46:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :mime-version:content-type:content-disposition:user-agent; bh=V9NkrMxh8PZ/t3CUj65wzbHouBSrNzROIKw3/DlyymY=; b=AZvgz2WVD84sdWMmDOFFrtrsKT7ojA3wK2aRw0DPYp9NUVIsqm3IbS/gdXnVK9e3uq R5CMfBL3MZUeidIA9XN1mjoyGdXi66hIT4fxkjRRmbxf4g6rwoxfVoVIaRJhz+XSVaOf pK2SjEaR8rDuziBfa1QuZxYJLQUlHtLyoBVC14w9FW/hVqrBsKOuA12IshuqI24LPZPx tTZ0Eqzjbi5UdrdupsE/ZG1clD8TzTOfBI8lVzFXUrtJ1F+onJ4GvvuVFE1bSEn+BDXM MPt94laZ1pojJBJeL8h+dP+OzD20hBbo+eaUxLueIEj4MLToOw48If1UQ8eepje1YXZo 9zWg== X-Received: by 10.68.224.133 with SMTP id rc5mr18961331pbc.80.1438983993683; Fri, 07 Aug 2015 14:46:33 -0700 (PDT) Received: from nparhar-pc (nat-198-95-226-228.netapp.com. [198.95.226.228]) by smtp.gmail.com with ESMTPSA id qb10sm11279419pab.39.2015.08.07.14.46.31 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Aug 2015 14:46:32 -0700 (PDT) Sender: Navdeep Parhar Date: Fri, 7 Aug 2015 14:46:26 -0700 From: Navdeep Parhar To: hps@selasky.org Cc: freebsd-infiniband@FreeBSD.org Subject: unify_tcp_port_space in OFED Message-ID: <20150807214626.GA3146@nparhar-pc> Mail-Followup-To: hps@selasky.org, freebsd-infiniband@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2015 21:46:34 -0000 Hello Hans, Looks like the last major OFED update (r278886) is missing unify_tcp_port_space and associated code in cma.c. See the same file in stable/10 or any revision prior to r278886. iWARP on FreeBSD has always used the kernel's TCP port space because that's the correct thing to do. I can fix this in head but if this piece of code is gone from upstream then we'll have to be extra careful during future OFED refreshes as we do want to keep this functionality in FreeBSD. Regards, Navdeep From owner-freebsd-infiniband@freebsd.org Fri Aug 7 23:37:54 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EA6E9B64DC for ; Fri, 7 Aug 2015 23:37:54 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1B335F6 for ; Fri, 7 Aug 2015 23:37:53 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by pdrg1 with SMTP id g1so50280967pdr.2 for ; Fri, 07 Aug 2015 16:37:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type; bh=CWHL6vRW4fz8Nc95fkf2z9ptxtcUiwn5B3rLy8cruZA=; b=SnzrM6MG65pWA4N9SVfXqDVe1TMid+RPxw7lmijPKTXfyk+tNzQSjk03VRDHt4S6Id +VsK9rwvTiD5F98YCWkv/T9Lxu1yC4IaSoQgbj1Kc3jXCZ/7EDPwcNNLGEJQzEC0dHjp 33yTpBJQ1Ql9+JbbebEsuA+Bj0YQ4PGEygY5Rj5l397oJRDNJfwANiT46VxyOQUtYk3P 3iDFl2HGv9XVq3swLteG8ibr8+l00mEFzPuAlyk6ChT+HP95I5MLkKX7lfiVaQEHBi+o 0vzuSVg6YcBF3JYa2SyjqDa7hsajYj7LadXIn52/sAc3DdxmQZCEIlmS2gF2l/KCaGjp IMFQ== X-Gm-Message-State: ALoCoQmtvrAHnGtkKMFEdE+BUoJoVo9Gm9vKZeiNfAYIHnlnj5BQ7rkWrPSm7D+Lcduh6T9E8Y2Z X-Received: by 10.70.24.33 with SMTP id r1mr19581606pdf.20.1438990667673; Fri, 07 Aug 2015 16:37:47 -0700 (PDT) Received: from rrcs-66-91-135-210.west.biz.rr.com (rrcs-66-91-135-210.west.biz.rr.com. [66.91.135.210]) by smtp.gmail.com with ESMTPSA id kk1sm11320221pbd.60.2015.08.07.16.37.45 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Aug 2015 16:37:46 -0700 (PDT) Date: Fri, 7 Aug 2015 13:37:01 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Mark Johnston cc: Garrett Cooper , freebsd-infiniband@FreeBSD.org, Benno Rice , Hans Petter Selasky Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? In-Reply-To: <20150807192930.GA88925@muskytusk> Message-ID: References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> <20150807192930.GA88925@muskytusk> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2015 23:37:54 -0000 On Fri, 7 Aug 2015, Mark Johnston wrote: > On Fri, Aug 07, 2015 at 11:32:00AM -0700, Garrett Cooper wrote: >> Hi, >> One of the complaints from engineers at Isilon I?ve received in the past is that Infiniband/OFED stack support isn?t enabled by default in GENERIC. I would like to enable it by default in GENERIC to improve test coverage by a general audience and ensure that bugs introduced elsewhere (build bugs, network interface, kernel interface bugs) aren?t ignored by accident when running make tinderbox builds as it?s not built by default. > > make tinderbox will build LINT kernels, which for amd64 will include the > OFED stack. > > As Jason pointed out, all of the IB stack (including the Linux compat > shims) can already be built as a KLD. Why not just make WITH_OFED the > default on amd64 instead? That way the KLDs and userland tools will be > built by default, and the size of the kernel needn't grow. > >> I?m considering just doing this for amd64, but I could be convinced to do i386 and other architectures; I can?t guarantee that it will be usable on arm*, mips, powerpc, sparc64, etc though. > I didn't do it originally because some tiny fraction of users will need it and it's not an 'out of the box' kind of feature. I do like developers not breaking it. Tinderbox building LINT seems most sane to me but I wouldn't weigh in strongly on this. Thanks, Jeff From owner-freebsd-infiniband@freebsd.org Sat Aug 8 05:38:48 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B9639B6C9F for ; Sat, 8 Aug 2015 05:38:48 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-qg0-x236.google.com (mail-qg0-x236.google.com [IPv6:2607:f8b0:400d:c04::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C4D6AF8; Sat, 8 Aug 2015 05:38:48 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by qgj62 with SMTP id 62so63130272qgj.2; Fri, 07 Aug 2015 22:38:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jwBbVwkOGSsB+UAX6/sAB+Y1B5Yw5uKYZo1Zun72TAE=; b=C08k7KXdHzwQsdDjCHAsmdz418rxT4QVM2LCbGqMl6nadiKdPQ9LCP0/HgOak3ZP4m izNsuBnVxfX7AdZHVwclaVg/Mxi3bzUSyfngRzX00QoBPHbzwHagwxw5Oag6dAeqrUL0 NRj9uKmUqVWZTdR4CycNfJLTBem4QU7/qtOGahTFnKMsLng/p4B1vRKmrPihWi/zCPXq tqUR+fS6Y/dne08lB/ta6RAvpg0DEDYbN8C5uCO7pVk2zXdiQj+701ei+jqx98pWxvyp ro2GwWQLaYtmtDj4G1l29DBT2GxUXB2b+J9vXmlmZMxAn5iXEuc4ALOqRBr0yxUU9+FZ Qnmw== MIME-Version: 1.0 X-Received: by 10.140.194.133 with SMTP id p127mr21248642qha.38.1439012327175; Fri, 07 Aug 2015 22:38:47 -0700 (PDT) Received: by 10.140.98.73 with HTTP; Fri, 7 Aug 2015 22:38:47 -0700 (PDT) In-Reply-To: <55C502AA.4080408@selasky.org> References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> <55C4FCC2.9060704@tds.net> <55C502AA.4080408@selasky.org> Date: Fri, 7 Aug 2015 22:38:47 -0700 Message-ID: Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? From: NGie Cooper To: Hans Petter Selasky Cc: Jason Bacon , freebsd-infiniband@freebsd.org, Mark Johnston , Jeff Roberson , Benno Rice Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2015 05:38:48 -0000 On Fri, Aug 7, 2015 at 12:10 PM, Hans Petter Selasky wrote: > Hi, > > See here: > > https://reviews.freebsd.org/D3182 I'll look at that too. From owner-freebsd-infiniband@freebsd.org Sat Aug 8 05:42:45 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E06EE9B6DC3 for ; Sat, 8 Aug 2015 05:42:45 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E962D47; Sat, 8 Aug 2015 05:42:45 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by qkdv3 with SMTP id v3so43828131qkd.3; Fri, 07 Aug 2015 22:42:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=rbTGaAEVT0g2D2HySbpeKqfkx9vmxTOl7QEoSEKARug=; b=MQiK+DRMnXAm+ZHcGXFdb9no5Fi8IRDS/4o+5n/ulJlnuVp+5o9oGaui3v7+4/cfBo inDGpfuTdi2xBfVnUz1MBwAL8YQGvOrWIgp9S3BhNBVWIyhaukWa20vntjOO8eh4kjj+ H/MgelPHDfHL8VX+xilI0KcITc0mqziNQPgpuy5fKejE5siRjnzOXLXbCUfajNuJUnOr EXZCFOmyUR9uL010eVVRYjDLWWbWm38Jilo2gu9gqGCJtydgiRa3g5tFMoeWYjj36xrp Nd8hO/Bw0ogE9O/Rje7Lmdo3lS78RUNXVxkEbeK0fGxc1GXnjp+Zir1371qE8Bj9yqlP jyOQ== MIME-Version: 1.0 X-Received: by 10.55.31.96 with SMTP id f93mr20387070qkf.5.1439012564665; Fri, 07 Aug 2015 22:42:44 -0700 (PDT) Received: by 10.140.98.73 with HTTP; Fri, 7 Aug 2015 22:42:44 -0700 (PDT) In-Reply-To: <20150807192930.GA88925@muskytusk> References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> <20150807192930.GA88925@muskytusk> Date: Fri, 7 Aug 2015 22:42:44 -0700 Message-ID: Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? From: NGie Cooper To: Mark Johnston Cc: freebsd-infiniband@freebsd.org, Benno Rice , Hans Petter Selasky , Jeff Roberson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2015 05:42:46 -0000 On Fri, Aug 7, 2015 at 12:29 PM, Mark Johnston wrote: > On Fri, Aug 07, 2015 at 11:32:00AM -0700, Garrett Cooper wrote: >> Hi, >> One of the complaints from engineers at Isilon I=E2=80=99ve receiv= ed in the past is that Infiniband/OFED stack support isn=E2=80=99t enabled = by default in GENERIC. I would like to enable it by default in GENERIC to i= mprove test coverage by a general audience and ensure that bugs introduced = elsewhere (build bugs, network interface, kernel interface bugs) aren=E2=80= =99t ignored by accident when running make tinderbox builds as it=E2=80=99s= not built by default. > > make tinderbox will build LINT kernels, which for amd64 will include the > OFED stack. > > As Jason pointed out, all of the IB stack (including the Linux compat > shims) can already be built as a KLD. Why not just make WITH_OFED the > default on amd64 instead? That way the KLDs and userland tools will be > built by default, and the size of the kernel needn't grow. There are a few issues with just doing WITH_OFED, instead of building both the sys/ofed and contrib/ofed separately: 1. sys/ofed by itself isn't incredibly useful (as we've seen internally). Yes, not building opensm can be done if you're running it on an IB switch, but the diagnostic tools are pretty helpful.. 2. contrib/ofed has seen its fair share of bugs in the past compilation wise (either due to interfaces or general header compilation issues). I'd rather nip these in the bud ASAP instead of delay them. 3. Building it just on amd64 might disguise issues with endianness, 64-bit issues, etc. Again, I want opensm, etc to be as useful on all platforms, if possible. Thanks! -NGie From owner-freebsd-infiniband@freebsd.org Sat Aug 8 06:54:05 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D25099ACC29 for ; Sat, 8 Aug 2015 06:54:05 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A76BAFEE for ; Sat, 8 Aug 2015 06:54:05 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by pacrr5 with SMTP id rr5so67208524pac.3 for ; Fri, 07 Aug 2015 23:53:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type; bh=QotNDwCroUE+xJ64eXWlrDJmSgh8LXQ/Ed9eITrMnTI=; b=dKtG5w8xfQsH7qi+LCtzeFozIGRTE2WH2FaaTrsbkNuZDOCpD9tZaolsGOH0ripUno Ods9ETYjUtNqaFfboaYcmQ6wlXjsS1Yos8Gnj/Gyq9EQsHoFJNAdsGQ/Iu6deSLDKpqh z2yzAhtx/OQ1zfuCTAsi8ggBB0TqsTA6LOKnbVou2gxDW+eRu2rqFHZoddkvKF93amx7 XvZ8cnF0Md1+IyIX86TVr9NaiFS7GuteHNCk1vZhjZQjxz0UV0mou/bsiP/qieuUaW2S kGBL4q/MCgtU4YmxNa9xFe7GSgiVqudW0UUqxgXlztDE2Kif8U8Yotv145fnSkZblhbw S2dw== X-Gm-Message-State: ALoCoQmbB7DuW5FOnq1BFidSbNfG655ISzOpolyk54wnbjWJ66vmBezezGr4StE/dVbY8yOlrEsC X-Received: by 10.66.253.170 with SMTP id ab10mr23332451pad.135.1439016839507; Fri, 07 Aug 2015 23:53:59 -0700 (PDT) Received: from rrcs-66-91-135-210.west.biz.rr.com (rrcs-66-91-135-210.west.biz.rr.com. [66.91.135.210]) by smtp.gmail.com with ESMTPSA id t15sm8500985pbs.10.2015.08.07.23.53.57 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Aug 2015 23:53:58 -0700 (PDT) Date: Fri, 7 Aug 2015 20:53:12 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: NGie Cooper cc: Mark Johnston , freebsd-infiniband@freebsd.org, Benno Rice , Hans Petter Selasky Subject: Re: Enable OFED/Infiniband support in 11.0-RELEASE by default? In-Reply-To: Message-ID: References: <30977F3A-59D1-4CA4-BCF6-9062A04CFF44@gmail.com> <20150807192930.GA88925@muskytusk> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2015 06:54:06 -0000 On Fri, 7 Aug 2015, NGie Cooper wrote: > On Fri, Aug 7, 2015 at 12:29 PM, Mark Johnston wrote: >> On Fri, Aug 07, 2015 at 11:32:00AM -0700, Garrett Cooper wrote: >>> Hi, >>> One of the complaints from engineers at Isilon I?ve received in the past is that Infiniband/OFED stack support isn?t enabled by default in GENERIC. I would like to enable it by default in GENERIC to improve test coverage by a general audience and ensure that bugs introduced elsewhere (build bugs, network interface, kernel interface bugs) aren?t ignored by accident when running make tinderbox builds as it?s not built by default. >> >> make tinderbox will build LINT kernels, which for amd64 will include the >> OFED stack. >> >> As Jason pointed out, all of the IB stack (including the Linux compat >> shims) can already be built as a KLD. Why not just make WITH_OFED the >> default on amd64 instead? That way the KLDs and userland tools will be >> built by default, and the size of the kernel needn't grow. > > There are a few issues with just doing WITH_OFED, instead of building > both the sys/ofed and contrib/ofed separately: > 1. sys/ofed by itself isn't incredibly useful (as we've seen > internally). Yes, not building opensm can be done if you're running it > on an IB switch, but the diagnostic tools are pretty helpful.. > 2. contrib/ofed has seen its fair share of bugs in the past > compilation wise (either due to interfaces or general header > compilation issues). I'd rather nip these in the bud ASAP instead of > delay them. > 3. Building it just on amd64 might disguise issues with endianness, > 64-bit issues, etc. Again, I want opensm, etc to be as useful on all > platforms, if possible. At least when I originally did the port, architectures other than x86/amd64 were not supported because I couldn't find a straightforward way to wrap the linux address handling in a way that was compatible with busdma. Jeff > Thanks! > -NGie > From owner-freebsd-infiniband@freebsd.org Sat Aug 8 08:48:19 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EBFD9B352B for ; Sat, 8 Aug 2015 08:48:19 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 258E71D8C for ; Sat, 8 Aug 2015 08:48:18 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 9CF871FE023; Sat, 8 Aug 2015 10:48:15 +0200 (CEST) Message-ID: <55C5C2A8.4070908@selasky.org> Date: Sat, 08 Aug 2015 10:49:44 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-infiniband@FreeBSD.org, Oded Shanoon Subject: Re: unify_tcp_port_space in OFED References: <20150807214626.GA3146@nparhar-pc> In-Reply-To: <20150807214626.GA3146@nparhar-pc> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2015 08:48:19 -0000 On 08/07/15 23:46, Navdeep Parhar wrote: > Hello Hans, > > Looks like the last major OFED update (r278886) is missing unify_tcp_port_space > and associated code in cma.c. See the same file in stable/10 or any revision > prior to r278886. iWARP on FreeBSD has always used the kernel's TCP port space > because that's the correct thing to do. I can fix this in head but if this > piece of code is gone from upstream then we'll have to be extra careful during > future OFED refreshes as we do want to keep this functionality in FreeBSD. > > Regards, > Navdeep > Hi Oded, Can you comment on the above? --HPS