From owner-freebsd-hackers@freebsd.org Wed Mar 1 17:48:11 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E01AACF2DF8 for ; Wed, 1 Mar 2017 17:48:11 +0000 (UTC) (envelope-from sauravsachidanand@gmail.com) Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 731E1C87; Wed, 1 Mar 2017 17:48:11 +0000 (UTC) (envelope-from sauravsachidanand@gmail.com) Received: by mail-wm0-x244.google.com with SMTP id m70so8593052wma.1; Wed, 01 Mar 2017 09:48:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ajiVKKAF4Uh2bfslfU738431XpGmmSlq22i43EXORew=; b=XSAjs+N0zggvq5kywlyfSgw/pPsWo0Q7WF1F7EKqF34XC95FsLkJ7tkcbYSqSozHju JiJRe/23bWIZtFGv4ezMC0Vl73KIu0l92d1KdS7ND3AxSl30r3tR1C7Wpi2oxJgnY+45 unQks+HwIY8vwjgITiBUO8PLDg24Wi1t8KfIOB9w+kR0bBKrOj3YxRPw4a7p1dz+DbTm 7P4fVu/uO3o26+iw1FnhBkionPHYlYT201LFaT609hwhodCrG2ux/EyLG0Sh2cOnv0ds occW5HFtYBU6HHWzI6mbzS9depkjUhM24aACgIqgk5BNY24SRcIR0EiTWYfxaxUfcdli wR3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ajiVKKAF4Uh2bfslfU738431XpGmmSlq22i43EXORew=; b=O32grDLrMkP5zSFDTD/L42rA4aKpob0bdygkVLu1MFrbduzoAbEz2MWY6mg0mhy/lS emMNlnCG1JFBT8rGHBBoO2Ehq8FEd45tyIomdJnsjm1hN9figW1jIl8CpljwcMIMSAKQ 7xFmqzZA9TZeS80ztb7PXiLgrGhC+I7MqmNkxkfeqFXQqc1hIOcvTYMIl4NIwp0LIYE1 xb+FibCL1iaU2HTANp3GwYWqVAZqVgEBR6Hl0JaxpOQa5PSoLobL8QHwoBsf6cipkbAB 1d6FYjiVVO1DP9PG2ys2RRUgxl7ci+8+VSwUDBiCYSDrNFjaJ8+drbXJX5XZg2/0JJSi LF0Q== X-Gm-Message-State: AMke39ld40fQSFhoY2zfm1b5sU9YpU9ceXChC3iIhOZowA3aw2AsvqCqRXic6dfDs2HtdenJjJDr3BhMFWttXg== X-Received: by 10.28.178.142 with SMTP id b136mr4402063wmf.57.1488390489608; Wed, 01 Mar 2017 09:48:09 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.144.6 with HTTP; Wed, 1 Mar 2017 09:48:09 -0800 (PST) In-Reply-To: References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <1488383213.60166.10.camel@freebsd.org> From: Saurav Sachidanand Date: Wed, 1 Mar 2017 23:18:09 +0530 Message-ID: Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Alan Somers Cc: Warner Losh , "freebsd-hackers@freebsd.org" , Allan Jude , Ian Lepore Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Mar 2017 17:48:12 -0000 Since the language(s) to be included are chosen at compile time it wouldn't be forcing anyone to use a bloated bootloader. Wouldn't Python's richer standard library (eg: re) be useful in certain situations? On Wed, Mar 1, 2017 at 9:42 PM, Alan Somers wrote: > Python is huge, and we already have people complaining that the > bootloader doesn't fit in their boot partitions ever since it gained > GELI support. The advantage of Lua is that the embedded interpreter > is very small. I'd rather not bloat the bootloader too much. > > -Alan > > On Wed, Mar 1, 2017 at 9:08 AM, Saurav Sachidanand > wrote: >> Is it related to this project >> https://wiki.freebsd.org/SummerOfCode2014/LuaLoader ? >> Apparently, that project created a generic interface in the bootloader >> to plug in any >> interpreter, and then added Lua. >> >> How about adding Python as well, as a GSoC project? A team from Intel managed to >> get Python to run inside GRUB [1] [2]. I can use their work as a >> reference for modifying >> the Python interpreter. >> >> [1] - https://lwn.net/Articles/641244/ >> [2] - https://github.com/biosbits/bits >> >> On Wed, Mar 1, 2017 at 3:51 PM, Warner Losh wrote: >>> On Wed, Mar 1, 2017 at 8:46 AM, Ian Lepore wrote: >>>> On Wed, 2017-03-01 at 16:14 +0800, Julian Elischer wrote: >>>>> On 28/2/17 2:01 am, Warner Losh wrote: >>>>> > >>>>> > On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude >>>>> > wrote: >>>>> > > >>>>> > > On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand >>>> > > chidanand@gmail.com> wrote: >>>>> > > > >>>>> > > > Hello FreeBSD community, >>>>> > > > >>>>> > > > I'm >>>>> > > > Saurav Sachidanand, and I'm >>>>> > > > a CS sophomore studying in India >>>>> > > > . >>>>> > > > I have an interest in operating systems development and wish to >>>>> > > > contribute >>>>> > > > to the FreeBSD community. I'm proficient with C and have some >>>>> > > > experience in >>>>> > > > kernel programming. Hence, I'd like to propose an original >>>>> > > > project for >>>>> > > > GSoC >>>>> > > > 2017 that I feel would benefit this community. >>>>> > > > >>>>> > > > In past years, the Lua interpreter was ported to run inside the >>>>> > > > Linux >>>>> > > > and >>>>> > > > NetBSD kernel [1]. Lua was chosen because it's interpreter is >>>>> > > > very >>>>> > > > small (~240 >>>>> > > > KB) compared to that of Python or Ruby, it's MIT licensed, and >>>>> > > > is >>>>> > > > almost >>>>> > > > freestanding. A working demonstration of it is a packet >>>>> > > > filtering >>>>> > > > algorithm >>>>> > > > written entirely in kernel Lua [2]. >>>>> > > > >>>>> > > > Specifically, my proposal would be to port the following that >>>>> > > > are >>>>> > > > currently >>>>> > > > written for NetBSD: >>>>> > > > - the modified Lua VM source code with _KERNEL preprocessor >>>>> > > > directives >>>>> > > > to >>>>> > > > exclude user-space functionality like floating point, the io >>>>> > > > and os >>>>> > > > module >>>>> > > > in the standard library, etc. [3] >>>>> > > > - the kernel module device driver for /dev/lua, to which Lua >>>>> > > > scripts >>>>> > > > are >>>>> > > > fed to be executed [4], [5] >>>>> > > > - the luactl user-space program to control the Lua device and a >>>>> > > > couple >>>>> > > > of >>>>> > > > sysctl variables which serve similar purpose [6], [7] >>>>> > > > >>>>> > > > And then: >>>>> > > > - run the Lua test suite targeting whatever we support in the >>>>> > > > kernel to >>>>> > > > make sure it works [8] >>>>> > > > - and write Lua bindings to the kernel interfaces that would >>>>> > > > interest >>>>> > > > the >>>>> > > > FreeBSD community >>>>> > > > >>>>> > > > Since NetBSD and FreeBSD have similar kernel interfaces >>>>> > > > (mutexes, >>>>> > > > linked >>>>> > > > lists, device switch interface), the porting shouldn't involve >>>>> > > > too much >>>>> > > > code refactoring. Also, this would all be an experiment in that >>>>> > > > we >>>>> > > > don't >>>>> > > > fully know what the real world use cases might be, but it would >>>>> > > > attract >>>>> > > > more people to writing kernel code who otherwise wouldn't >>>>> > > > because of >>>>> > > > having >>>>> > > > to do everything in C. And it would be interesting to carry out >>>>> > > > it out >>>>> > > > in >>>>> > > > FreeBSD as well since it has a larger community than NetBSD. >>>>> > > > >>>>> > > > I humbly request anyone who is interested in this project to be >>>>> > > > my >>>>> > > > potential mentor(s) for GSoC. >>>>> > > > >>>>> > > > More slides on kernel Lua in NetBSD - [9], [10]. >>>>> > > > >>>>> > > > Thanks, >>>>> > > > Saurav >>>>> > > > >>>>> > > > [1] - http://www.netbsd.org/~lneto/dls14.pdf >>>>> > > > [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf >>>>> > > > [3] - https://github.com/jsonn/src/tree/trunk/external/mit/lua/ >>>>> > > > dist/src >>>>> > > > [4] - >>>>> > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/module >>>>> > > > s/lua >>>>> > > > [5] - >>>>> > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/module >>>>> > > > s/luasystm >>>>> > > > [6] - https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sbin >>>>> > > > /luactl >>>>> > > > [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetBSD-current >>>>> > > > [8] - http://www.lua.org/tests/ >>>>> > > > [9] - >>>>> > > > https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2012 >>>>> > > > /kernel_mode_lua.pdf >>>>> > > > [10] - https://www.lua.org/wshop13/Cormack.pdf >>>>> > > > _______________________________________________ >>>>> > > > freebsd-hackers@freebsd.org mailing list >>>>> > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>> > > > To unsubscribe, send any mail to >>>>> > > > "freebsd-hackers-unsubscribe@freebsd.org" >>>>> > > This may be quite a nice thing to have. Another upcoming use for >>>>> > > LUA in the kernel is ZFS Channel Programs. These allow a number >>>>> > > of ZFS operations to be completed as a single atomic transaction. >>>>> > > >>>>> > > I would hope we could structure this in such a way as to not end >>>>> > > up with two copies of Lua in the kernel. >>>>> > There's also a 3/4 finished lua in the boot loader that you might >>>>> > be >>>>> > able to leverage as well.... >>>>> I'd like to see that finished. While Devin has done Heroic work with >>>>> the forth in the loader, I think it's time has come. >>>>> It' be nice to have something a little less '60s. >>>>> >>>>> > >>>>> > >>>>> > Warner >>>> >>>> I was under the impression that the "lua in bootloader" work was >>>> basically done and just needed testing, which nobody has done. I think >>>> it's all sitting in the projects/lua-bootloader branch in svn. >>> >>> The branch compiles. Testing has been done, but there's some missing >>> bits. It basically kinda works for the average case, but more advanced >>> uses of the bootloader still have sharp pointy edges on them, the >>> extent of the pointy edges is unknown. At this point the rebasing of >>> the branch is non-trivial due to the merge conflicts that have crept >>> in. They don't look awful, but when I tried to use git to rebase to a >>> more modern FreeBSD, there were lots of stupid things. Maybe I'll try >>> again... >>> >>> Warner >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"