From owner-freebsd-fs@freebsd.org Thu Apr 14 21:39:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B87AAEC0DF for ; Thu, 14 Apr 2016 21:39:07 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E5F8E1516 for ; Thu, 14 Apr 2016 21:39:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:Ig8UMR+PkLFK/v9uRHKM819IXTAuvvDOBiVQ1KB91u4cTK2v8tzYMVDF4r011RmSDdWdtq0P07SempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXsq3G/pQQfBg/4fVIsYL+lSsiP0I/viaibwN76XUZhvHKFe7R8LRG7/036l/I9ps9cEJs30QbDuXBSeu5blitCLFOXmAvgtI/rpMYwu3cYh/V0+dRNXK/hc+E2VvQMAi4rPmou6IjlrjHNVwaC7GAQFGIMnUwbLRLC6UTAX5zy+g7zvel51SzSadfzRLs3XTmnx7psRwLljD8HcTUwpjKEwvdshb5W9Ury7yd0xJTZNdmY X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CqBACPDBBX/61jaINeFoN1fQa8JyKFbIF+EQEBAQEBAQEBZCeCLYIbIwRNFwEiAg0ZAlsEiDwOoD+PXZJOfIUlgX2GdYMYglYFh3aQFYV3iX1OhyiFM4YhiQYCNiyBQoJBIDABiEd+AQEB X-IronPort-AV: E=Sophos;i="5.24,485,1454994000"; d="scan'208";a="277923504" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 14 Apr 2016 17:37:58 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 1249B15F58F for ; Thu, 14 Apr 2016 17:37:58 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id fpZws5Xq28fz for ; Thu, 14 Apr 2016 17:37:57 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 8160A15F592 for ; Thu, 14 Apr 2016 17:37:57 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4w0UeYK-0CkV for ; Thu, 14 Apr 2016 17:37:57 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 673D915F58F for ; Thu, 14 Apr 2016 17:37:57 -0400 (EDT) Date: Thu, 14 Apr 2016 17:37:57 -0400 (EDT) From: Rick Macklem To: freebsd-fs Message-ID: <194021987.63592678.1460669877291.JavaMail.zimbra@uoguelph.ca> Subject: pNFS server for testing (alpha test stage) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF18 (Linux)/8.0.9_GA_6191) Thread-Topic: pNFS server for testing (alpha test stage) Thread-Index: j88m+u+umUjcwt3tkit07m5esbvbew== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Apr 2016 21:39:07 -0000 Hi, I've been working on a pNFS server for FreeBSD over the last few months and it is now at what I would consider alpha test. One thing I know is currently broken is appending to files (which sometimes get truncated). I'm sure there are lots of others. The dream is that someday this will allow a FreeBSD NFS service to scale beyond what a single server configuration can handle. Performance could be described as slow->very slow with the exception of reading/writing, which works at wire speed on my slow hardware (100Mbit/sec). Hopefully some of this will be resolved over time, but I'd guess that things like file creation will always be slow compared to doing NFS against a file system like ZFS. I will continue to work on it and update the project on svn but, to be honest, it won't go much further unless others jump "on the bandwagon" to do testing on larger hardware configurations than I have. Essentially this server consists of a GlusterFS cluster and uses GlusterFS's NFSv3 server to implement the data server component (read/write of files). Since the NFSv3 Read/Write RPCs are directed to the system in the GlusterFS cluster that the file exists on, it is hoped that this will scale. Multiple metadata servers (NFSv4.1 servers that do everything except reading/writing) may be possible, but that is a ways off. At this time I have only tested against the modified NFSv4.1 client in the projects area for FreeBSD. I hope to start testing against a Linux client sometime soon. (It uses a pNFS layout scheme called "Flexible File Layout" which is still at the internet draft stage.) Anyhow, if anyone is interested in testing this, I have a primitive document at: http://people.freebsd.org/~rmacklem/pnfs-setup.txt that will hopefully give you a starting point and, as this file notes, the modified sources are in FreeBSD's subversion repository in base/projects/pnfs-server. Have fun with it, if you try it, rick