From owner-freebsd-hackers@freebsd.org Wed Mar 1 17:49:20 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9291CF2F23 for ; Wed, 1 Mar 2017 17:49:20 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40DE3E1F for ; Wed, 1 Mar 2017 17:49:20 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 66536b32-fea7-11e6-95b5-6dfd7dbb0ee5 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id 66536b32-fea7-11e6-95b5-6dfd7dbb0ee5; Wed, 01 Mar 2017 17:49:22 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v21HnCjI007063; Wed, 1 Mar 2017 10:49:12 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1488390552.60166.23.camel@freebsd.org> Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD From: Ian Lepore To: Saurav Sachidanand , Alan Somers Cc: "freebsd-hackers@freebsd.org" , Allan Jude Date: Wed, 01 Mar 2017 10:49:12 -0700 In-Reply-To: References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <1488383213.60166.10.camel@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Mar 2017 17:49:20 -0000 The thought of python being any part of the base system is nothing less than horrifying.  Didn't we learn our lesson with perl all those years ago? -- Ian On Wed, 2017-03-01 at 23:18 +0530, Saurav Sachidanand wrote: > Since the language(s) to be included are chosen at compile time > it wouldn't be forcing anyone to use a bloated bootloader. > Wouldn't Python's richer standard library (eg: re) be useful in > certain situations? > > On Wed, Mar 1, 2017 at 9:42 PM, Alan Somers > wrote: > > > > Python is huge, and we already have people complaining that the > > bootloader doesn't fit in their boot partitions ever since it > > gained > > GELI support.  The advantage of Lua is that the embedded > > interpreter > > is very small.  I'd rather not bloat the bootloader too much. > > > > -Alan > > > > On Wed, Mar 1, 2017 at 9:08 AM, Saurav Sachidanand > > wrote: > > > > > > Is it related to this project > > > https://wiki.freebsd.org/SummerOfCode2014/LuaLoader ? > > > Apparently, that project created a generic interface in the > > > bootloader > > > to plug in any > > > interpreter, and then added Lua. > > > > > > How about adding Python as well, as a GSoC project? A team from > > > Intel managed to > > > get Python to run inside GRUB [1] [2]. I can use their work as a > > > reference for modifying > > > the Python interpreter. > > > > > > [1] - https://lwn.net/Articles/641244/ > > > [2] - https://github.com/biosbits/bits > > > > > > On Wed, Mar 1, 2017 at 3:51 PM, Warner Losh > > > wrote: > > > > > > > > On Wed, Mar 1, 2017 at 8:46 AM, Ian Lepore > > > > wrote: > > > > > > > > > > On Wed, 2017-03-01 at 16:14 +0800, Julian Elischer wrote: > > > > > > > > > > > > On 28/2/17 2:01 am, Warner Losh wrote: > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude > > > > > > eebsd.org> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand > > > > > > > > > > > > > > > chidanand@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello FreeBSD community, > > > > > > > > > > > > > > > > > > I'm > > > > > > > > > Saurav Sachidanand, and I'm > > > > > > > > > a CS sophomore studying in India > > > > > > > > > . > > > > > > > > > I have an interest in operating systems development > > > > > > > > > and wish to > > > > > > > > > contribute > > > > > > > > > to the FreeBSD community. I'm proficient with C and > > > > > > > > > have some > > > > > > > > > experience in > > > > > > > > > kernel programming. Hence, I'd like to propose an > > > > > > > > > original > > > > > > > > > project for > > > > > > > > > GSoC > > > > > > > > > 2017 that I feel would benefit this community. > > > > > > > > > > > > > > > > > > In past years, the Lua interpreter was ported to run > > > > > > > > > inside the > > > > > > > > > Linux > > > > > > > > > and > > > > > > > > > NetBSD kernel [1]. Lua was chosen because it's > > > > > > > > > interpreter is > > > > > > > > > very > > > > > > > > > small (~240 > > > > > > > > > KB) compared to that of Python or Ruby, it's MIT > > > > > > > > > licensed, and > > > > > > > > > is > > > > > > > > > almost > > > > > > > > > freestanding. A working demonstration of it is a > > > > > > > > > packet > > > > > > > > > filtering > > > > > > > > > algorithm > > > > > > > > > written entirely in kernel Lua [2]. > > > > > > > > > > > > > > > > > > Specifically, my proposal would be to port the > > > > > > > > > following that > > > > > > > > > are > > > > > > > > > currently > > > > > > > > > written for NetBSD: > > > > > > > > > - the modified Lua VM source code with _KERNEL > > > > > > > > > preprocessor > > > > > > > > > directives > > > > > > > > > to > > > > > > > > > exclude user-space functionality like floating point, > > > > > > > > > the io > > > > > > > > > and os > > > > > > > > > module > > > > > > > > > in the standard library, etc. [3] > > > > > > > > > - the kernel module device driver for /dev/lua, to > > > > > > > > > which Lua > > > > > > > > > scripts > > > > > > > > > are > > > > > > > > > fed to be executed [4], [5] > > > > > > > > > - the luactl user-space program to control the Lua > > > > > > > > > device and a > > > > > > > > > couple > > > > > > > > > of > > > > > > > > > sysctl variables which serve similar purpose [6], [7] > > > > > > > > > > > > > > > > > > And then: > > > > > > > > > - run the Lua test suite targeting whatever we > > > > > > > > > support in the > > > > > > > > > kernel to > > > > > > > > > make sure it works [8] > > > > > > > > > - and write Lua bindings to the kernel interfaces > > > > > > > > > that would > > > > > > > > > interest > > > > > > > > > the > > > > > > > > > FreeBSD community > > > > > > > > > > > > > > > > > > Since NetBSD and FreeBSD have similar kernel > > > > > > > > > interfaces > > > > > > > > > (mutexes, > > > > > > > > > linked > > > > > > > > > lists, device switch interface), the porting > > > > > > > > > shouldn't involve > > > > > > > > > too much > > > > > > > > > code refactoring. Also, this would all be an > > > > > > > > > experiment in that > > > > > > > > > we > > > > > > > > > don't > > > > > > > > > fully know what the real world use cases might be, > > > > > > > > > but it would > > > > > > > > > attract > > > > > > > > > more people to writing kernel code who otherwise > > > > > > > > > wouldn't > > > > > > > > > because of > > > > > > > > > having > > > > > > > > > to do everything in C. And it would be interesting to > > > > > > > > > carry out > > > > > > > > > it out > > > > > > > > > in > > > > > > > > > FreeBSD as well since it has a larger community than > > > > > > > > > NetBSD. > > > > > > > > > > > > > > > > > > I humbly request anyone who is interested in this > > > > > > > > > project to be > > > > > > > > > my > > > > > > > > > potential mentor(s) for GSoC. > > > > > > > > > > > > > > > > > > More slides on kernel Lua in NetBSD - [9], [10]. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Saurav > > > > > > > > > > > > > > > > > > [1] - http://www.netbsd.org/~lneto/dls14.pdf > > > > > > > > > [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf > > > > > > > > > [3] - https://github.com/jsonn/src/tree/trunk/externa > > > > > > > > > l/mit/lua/ > > > > > > > > > dist/src > > > > > > > > > [4] - > > > > > > > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/ > > > > > > > > > sys/module > > > > > > > > > s/lua > > > > > > > > > [5] - > > > > > > > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/ > > > > > > > > > sys/module > > > > > > > > > s/luasystm > > > > > > > > > [6] - https://github.com/IIJ-NetBSD/netbsd-src/tree/m > > > > > > > > > aster/sbin > > > > > > > > > /luactl > > > > > > > > > [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetB > > > > > > > > > SD-current > > > > > > > > > [8] - http://www.lua.org/tests/ > > > > > > > > > [9] - > > > > > > > > > https://www.netbsd.org/gallery/presentations/mbalmer/ > > > > > > > > > fosdem2012 > > > > > > > > > /kernel_mode_lua.pdf > > > > > > > > > [10] - https://www.lua.org/wshop13/Cormack.pdf > > > > > > > > > _______________________________________________ > > > > > > > > > freebsd-hackers@freebsd.org mailing list > > > > > > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-ha > > > > > > > > > ckers > > > > > > > > > To unsubscribe, send any mail to > > > > > > > > > "freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > > This may be quite a nice thing to have. Another > > > > > > > > upcoming use for > > > > > > > > LUA in the kernel is ZFS Channel Programs. These allow > > > > > > > > a number > > > > > > > > of ZFS operations to be completed as a single atomic > > > > > > > > transaction. > > > > > > > > > > > > > > > > I would hope we could structure this in such a way as > > > > > > > > to not end > > > > > > > > up with two copies of Lua in the kernel. > > > > > > > There's also a 3/4 finished lua in the boot loader that > > > > > > > you might > > > > > > > be > > > > > > > able to leverage as well.... > > > > > > I'd like to see that finished. While Devin has done Heroic > > > > > > work with > > > > > > the forth in the loader, I think it's time has come. > > > > > > It' be nice to have something a little less '60s. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Warner > > > > > I was under the impression that the "lua in bootloader" work > > > > > was > > > > > basically done and just needed testing, which nobody has > > > > > done.  I think > > > > > it's all sitting in the projects/lua-bootloader branch in > > > > > svn. > > > > The branch compiles. Testing has been done, but there's some > > > > missing > > > > bits. It basically kinda works for the average case, but more > > > > advanced > > > > uses of the bootloader still have sharp pointy edges on them, > > > > the > > > > extent of the pointy edges is unknown. At this point the > > > > rebasing of > > > > the branch is non-trivial due to the merge conflicts that have > > > > crept > > > > in. They don't look awful, but when I tried to use git to > > > > rebase to a > > > > more modern FreeBSD, there were lots of stupid things. Maybe > > > > I'll try > > > > again... > > > > > > > > Warner > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@fre > > > ebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd > .org"