From owner-freebsd-hackers@freebsd.org Wed Mar 1 16:08:26 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C896CE4597 for ; Wed, 1 Mar 2017 16:08:26 +0000 (UTC) (envelope-from sauravsachidanand@gmail.com) Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 203BFC06; Wed, 1 Mar 2017 16:08:26 +0000 (UTC) (envelope-from sauravsachidanand@gmail.com) Received: by mail-wm0-x242.google.com with SMTP id m70so8151567wma.1; Wed, 01 Mar 2017 08:08:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fIayQCfIWZkPy1Wjsyu8VRnSHnP58fcLPdsrudQAccA=; b=tdmbudGxBDLhrSqHMwy/eI+ZG8IkLyqAp1Rg6MlOfiZB8J/VztbqAnJ5dTu12WAUEO pvOLiKXW/4H/1wVB6cbyNus1S7B7eLpvdn2x9SrbAUL6p+9utcdBgZKheBn5l4TKxifZ A9P1lvPsySi4SWVWo7+OVXN3CYrMbzwHCGgvtpRN0dNm4Hqr5srOCJZUYCdQi5WhWGdG /wmpzK1k0Qnq9DoQ9rElo9jAe/gchZpyPFKoK55d1DA9tvXC0KtY6VwyvgXLAVhgZITU T32mcQXMhJvmWmW1lGnndHVLkOurKBJwP4MIuptRnLcpxeAHX+d3CNcKRuc0R8K5VJuC +Q0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fIayQCfIWZkPy1Wjsyu8VRnSHnP58fcLPdsrudQAccA=; b=Lz+Xn4p7vlY+JCVT3JoSzJv2GUh/Dh+px4pA25DPft0GDFNUcTzjJc2v4N0gZBIMBJ kdKCyUqKCeN6Y+rOXweg9T9xg1x8pyfx+z8qf9TqbOzXbZvDU1UCgYTinBNDY7eH+UoA 0fIKVw9sQIdeXkbFLe13cK7TjGLYOf/cAuQyA7Segn7AToKRtxfAe0v8+6RF8fuVgui+ 5X1SmUIsKTrBFI+JC8vmyFvV7Nhoe4wlA7z2Ne6d5di4Gnjp4U/1sXZNfP6zqfIuHjET +junOpROqDjj4/goSJtXMTMhJpAn/B76obHmWp5+xG020cZ240Bk15ZCgkTYBeYv/Bul wueA== X-Gm-Message-State: AMke39mzsbUDL/aLgN3e+CtvAmpq1cD6eoex/z4rmVhDnTIsSoQhOKiJyh0SI1prle/Z54YEOq8hF4U7w0BsTA== X-Received: by 10.28.181.80 with SMTP id e77mr4024205wmf.57.1488384504269; Wed, 01 Mar 2017 08:08:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.144.6 with HTTP; Wed, 1 Mar 2017 08:08:23 -0800 (PST) In-Reply-To: References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <1488383213.60166.10.camel@freebsd.org> From: Saurav Sachidanand Date: Wed, 1 Mar 2017 16:08:23 +0000 Message-ID: Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Warner Losh Cc: Ian Lepore , Julian Elischer , Allan Jude , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Mar 2017 16:08:26 -0000 Is it related to this project https://wiki.freebsd.org/SummerOfCode2014/LuaLoader ? Apparently, that project created a generic interface in the bootloader to plug in any interpreter, and then added Lua. How about adding Python as well, as a GSoC project? A team from Intel managed to get Python to run inside GRUB [1] [2]. I can use their work as a reference for modifying the Python interpreter. [1] - https://lwn.net/Articles/641244/ [2] - https://github.com/biosbits/bits On Wed, Mar 1, 2017 at 3:51 PM, Warner Losh wrote: > On Wed, Mar 1, 2017 at 8:46 AM, Ian Lepore wrote: >> On Wed, 2017-03-01 at 16:14 +0800, Julian Elischer wrote: >>> On 28/2/17 2:01 am, Warner Losh wrote: >>> > >>> > On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude >>> > wrote: >>> > > >>> > > On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand >> > > chidanand@gmail.com> wrote: >>> > > > >>> > > > Hello FreeBSD community, >>> > > > >>> > > > I'm >>> > > > Saurav Sachidanand, and I'm >>> > > > a CS sophomore studying in India >>> > > > . >>> > > > I have an interest in operating systems development and wish to >>> > > > contribute >>> > > > to the FreeBSD community. I'm proficient with C and have some >>> > > > experience in >>> > > > kernel programming. Hence, I'd like to propose an original >>> > > > project for >>> > > > GSoC >>> > > > 2017 that I feel would benefit this community. >>> > > > >>> > > > In past years, the Lua interpreter was ported to run inside the >>> > > > Linux >>> > > > and >>> > > > NetBSD kernel [1]. Lua was chosen because it's interpreter is >>> > > > very >>> > > > small (~240 >>> > > > KB) compared to that of Python or Ruby, it's MIT licensed, and >>> > > > is >>> > > > almost >>> > > > freestanding. A working demonstration of it is a packet >>> > > > filtering >>> > > > algorithm >>> > > > written entirely in kernel Lua [2]. >>> > > > >>> > > > Specifically, my proposal would be to port the following that >>> > > > are >>> > > > currently >>> > > > written for NetBSD: >>> > > > - the modified Lua VM source code with _KERNEL preprocessor >>> > > > directives >>> > > > to >>> > > > exclude user-space functionality like floating point, the io >>> > > > and os >>> > > > module >>> > > > in the standard library, etc. [3] >>> > > > - the kernel module device driver for /dev/lua, to which Lua >>> > > > scripts >>> > > > are >>> > > > fed to be executed [4], [5] >>> > > > - the luactl user-space program to control the Lua device and a >>> > > > couple >>> > > > of >>> > > > sysctl variables which serve similar purpose [6], [7] >>> > > > >>> > > > And then: >>> > > > - run the Lua test suite targeting whatever we support in the >>> > > > kernel to >>> > > > make sure it works [8] >>> > > > - and write Lua bindings to the kernel interfaces that would >>> > > > interest >>> > > > the >>> > > > FreeBSD community >>> > > > >>> > > > Since NetBSD and FreeBSD have similar kernel interfaces >>> > > > (mutexes, >>> > > > linked >>> > > > lists, device switch interface), the porting shouldn't involve >>> > > > too much >>> > > > code refactoring. Also, this would all be an experiment in that >>> > > > we >>> > > > don't >>> > > > fully know what the real world use cases might be, but it would >>> > > > attract >>> > > > more people to writing kernel code who otherwise wouldn't >>> > > > because of >>> > > > having >>> > > > to do everything in C. And it would be interesting to carry out >>> > > > it out >>> > > > in >>> > > > FreeBSD as well since it has a larger community than NetBSD. >>> > > > >>> > > > I humbly request anyone who is interested in this project to be >>> > > > my >>> > > > potential mentor(s) for GSoC. >>> > > > >>> > > > More slides on kernel Lua in NetBSD - [9], [10]. >>> > > > >>> > > > Thanks, >>> > > > Saurav >>> > > > >>> > > > [1] - http://www.netbsd.org/~lneto/dls14.pdf >>> > > > [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf >>> > > > [3] - https://github.com/jsonn/src/tree/trunk/external/mit/lua/ >>> > > > dist/src >>> > > > [4] - >>> > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/module >>> > > > s/lua >>> > > > [5] - >>> > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/module >>> > > > s/luasystm >>> > > > [6] - https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sbin >>> > > > /luactl >>> > > > [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetBSD-current >>> > > > [8] - http://www.lua.org/tests/ >>> > > > [9] - >>> > > > https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2012 >>> > > > /kernel_mode_lua.pdf >>> > > > [10] - https://www.lua.org/wshop13/Cormack.pdf >>> > > > _______________________________________________ >>> > > > freebsd-hackers@freebsd.org mailing list >>> > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> > > > To unsubscribe, send any mail to >>> > > > "freebsd-hackers-unsubscribe@freebsd.org" >>> > > This may be quite a nice thing to have. Another upcoming use for >>> > > LUA in the kernel is ZFS Channel Programs. These allow a number >>> > > of ZFS operations to be completed as a single atomic transaction. >>> > > >>> > > I would hope we could structure this in such a way as to not end >>> > > up with two copies of Lua in the kernel. >>> > There's also a 3/4 finished lua in the boot loader that you might >>> > be >>> > able to leverage as well.... >>> I'd like to see that finished. While Devin has done Heroic work with >>> the forth in the loader, I think it's time has come. >>> It' be nice to have something a little less '60s. >>> >>> > >>> > >>> > Warner >> >> I was under the impression that the "lua in bootloader" work was >> basically done and just needed testing, which nobody has done. I think >> it's all sitting in the projects/lua-bootloader branch in svn. > > The branch compiles. Testing has been done, but there's some missing > bits. It basically kinda works for the average case, but more advanced > uses of the bootloader still have sharp pointy edges on them, the > extent of the pointy edges is unknown. At this point the rebasing of > the branch is non-trivial due to the merge conflicts that have crept > in. They don't look awful, but when I tried to use git to rebase to a > more modern FreeBSD, there were lots of stupid things. Maybe I'll try > again... > > Warner