From owner-freebsd-hackers@freebsd.org Wed Mar 1 17:49:56 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51AA9CF3012 for ; Wed, 1 Mar 2017 17:49:56 +0000 (UTC) (envelope-from bygrandao@gmail.com) Received: from mail-it0-x241.google.com (mail-it0-x241.google.com [IPv6:2607:f8b0:4001:c0b::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14ECCF28; Wed, 1 Mar 2017 17:49:56 +0000 (UTC) (envelope-from bygrandao@gmail.com) Received: by mail-it0-x241.google.com with SMTP id w185so5756908ita.3; Wed, 01 Mar 2017 09:49:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=dOjQ5aX3K5xKr+pOMtiGZROaQSRQxQdwYYS4bDTNfnU=; b=Wb8vxaP/9Ft/fJ1Ah78ose7jnkiTDSJYwEphNz9x+djMDjXDNAMwyfQ7Re9wV8lycp /Eqvd6WrUJbUIM5diFibhqfhtx51TvF+HZX5xaTWE4AqPUH2SYoqiq4WyDnFH9zSMWb9 IdSA4ZF6sex7JMQ1LGnRPIdmE0zxfL2YMvN/82YVxVTXRFaczFNFrpT1oduHMIzc5WfY fSBpW92n4ZdLFjp2Q9EaonlAzFRfvvkCpc7/YZUkZW43Ntsxwexjk5lxougeNH2VTmtr 03+FvCXPkGkY/4TDYGTIRaVFt1Vcxzu2vYGXGw8hugDqRKBj4EJyS+DRxYXeBNeaBZ5b F3nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=dOjQ5aX3K5xKr+pOMtiGZROaQSRQxQdwYYS4bDTNfnU=; b=XXzAzH20w9tFqJbwoZAWwBgIXjK+S7QUOpXn9nQIWr5qiBvTnsiP6KZsZMakjF6CgQ Gnu7hZvp8CEUjRLaL4mXaUrwoiJckcgSAX7dhGDMWVvEmVwtfVByqMt9S5Lr6rwZutro tKGcR/8HkQZAcp7q2uB8nyvRH5pST2FrQYdNqa2TeIGXtJJ5iKFftLP2xmkAaxXliAu8 31LckO1vQ9MBVAbBTBwv1EEjt9wzVphDWDK8hcbUaJjH4I7lOW5ESWEuytCHt3mnr6K6 QPYo9xUFgPLA6zEuAdJ8JV2JwS6lffzOMwrcWRZ7yhxebHoxCvAFXicpvg0MqrOledr/ FwXw== X-Gm-Message-State: AMke39kI/M8OE84QWLdySytEkSpTdzM7NxCt5zbiq5RZqNlDOs6Sx4JOcvbueNhn2wYY1ZDqzex1qhKej+bLHQ== X-Received: by 10.36.146.134 with SMTP id l128mr5587161itd.19.1488390595380; Wed, 01 Mar 2017 09:49:55 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.121.165 with HTTP; Wed, 1 Mar 2017 09:49:54 -0800 (PST) In-Reply-To: References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <1488383213.60166.10.camel@freebsd.org> From: Pedro Arthur Date: Wed, 1 Mar 2017 14:49:54 -0300 Message-ID: Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Alan Somers Cc: Saurav Sachidanand , "freebsd-hackers@freebsd.org" , Ian Lepore , Allan Jude Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Mar 2017 17:49:56 -0000 Hi, I'm the author of the lua bootloader project, I think Rui Paulo did a few improvements located in [1]. I don't remember correctly but I guess there was a few issues with lua using floating point for numbers, but it can easily be changed to use integers. I think rebasing the branch would be painful, maybe we could cherry pick the changes to a fresh branch? If needed I may help getting the code up to date, also it would be helpful if the "sharp edges" were documented in the project wiki page [2] or a new one dedicated to the lua bootloader. [1] - https://svnweb.freebsd.org/base/projects/lua-bootloader/ [2] - https://wiki.freebsd.org/SummerOfCode2014/LuaLoader 2017-03-01 13:12 GMT-03:00 Alan Somers : > Python is huge, and we already have people complaining that the > bootloader doesn't fit in their boot partitions ever since it gained > GELI support. The advantage of Lua is that the embedded interpreter > is very small. I'd rather not bloat the bootloader too much. > > -Alan > > On Wed, Mar 1, 2017 at 9:08 AM, Saurav Sachidanand > wrote: > > Is it related to this project > > https://wiki.freebsd.org/SummerOfCode2014/LuaLoader ? > > Apparently, that project created a generic interface in the bootloader > > to plug in any > > interpreter, and then added Lua. > > > > How about adding Python as well, as a GSoC project? A team from Intel > managed to > > get Python to run inside GRUB [1] [2]. I can use their work as a > > reference for modifying > > the Python interpreter. > > > > [1] - https://lwn.net/Articles/641244/ > > [2] - https://github.com/biosbits/bits > > > > On Wed, Mar 1, 2017 at 3:51 PM, Warner Losh wrote: > >> On Wed, Mar 1, 2017 at 8:46 AM, Ian Lepore wrote: > >>> On Wed, 2017-03-01 at 16:14 +0800, Julian Elischer wrote: > >>>> On 28/2/17 2:01 am, Warner Losh wrote: > >>>> > > >>>> > On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude > >>>> > wrote: > >>>> > > > >>>> > > On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand >>>> > > chidanand@gmail.com> wrote: > >>>> > > > > >>>> > > > Hello FreeBSD community, > >>>> > > > > >>>> > > > I'm > >>>> > > > Saurav Sachidanand, and I'm > >>>> > > > a CS sophomore studying in India > >>>> > > > . > >>>> > > > I have an interest in operating systems development and wish to > >>>> > > > contribute > >>>> > > > to the FreeBSD community. I'm proficient with C and have some > >>>> > > > experience in > >>>> > > > kernel programming. Hence, I'd like to propose an original > >>>> > > > project for > >>>> > > > GSoC > >>>> > > > 2017 that I feel would benefit this community. > >>>> > > > > >>>> > > > In past years, the Lua interpreter was ported to run inside the > >>>> > > > Linux > >>>> > > > and > >>>> > > > NetBSD kernel [1]. Lua was chosen because it's interpreter is > >>>> > > > very > >>>> > > > small (~240 > >>>> > > > KB) compared to that of Python or Ruby, it's MIT licensed, and > >>>> > > > is > >>>> > > > almost > >>>> > > > freestanding. A working demonstration of it is a packet > >>>> > > > filtering > >>>> > > > algorithm > >>>> > > > written entirely in kernel Lua [2]. > >>>> > > > > >>>> > > > Specifically, my proposal would be to port the following that > >>>> > > > are > >>>> > > > currently > >>>> > > > written for NetBSD: > >>>> > > > - the modified Lua VM source code with _KERNEL preprocessor > >>>> > > > directives > >>>> > > > to > >>>> > > > exclude user-space functionality like floating point, the io > >>>> > > > and os > >>>> > > > module > >>>> > > > in the standard library, etc. [3] > >>>> > > > - the kernel module device driver for /dev/lua, to which Lua > >>>> > > > scripts > >>>> > > > are > >>>> > > > fed to be executed [4], [5] > >>>> > > > - the luactl user-space program to control the Lua device and a > >>>> > > > couple > >>>> > > > of > >>>> > > > sysctl variables which serve similar purpose [6], [7] > >>>> > > > > >>>> > > > And then: > >>>> > > > - run the Lua test suite targeting whatever we support in the > >>>> > > > kernel to > >>>> > > > make sure it works [8] > >>>> > > > - and write Lua bindings to the kernel interfaces that would > >>>> > > > interest > >>>> > > > the > >>>> > > > FreeBSD community > >>>> > > > > >>>> > > > Since NetBSD and FreeBSD have similar kernel interfaces > >>>> > > > (mutexes, > >>>> > > > linked > >>>> > > > lists, device switch interface), the porting shouldn't involve > >>>> > > > too much > >>>> > > > code refactoring. Also, this would all be an experiment in that > >>>> > > > we > >>>> > > > don't > >>>> > > > fully know what the real world use cases might be, but it would > >>>> > > > attract > >>>> > > > more people to writing kernel code who otherwise wouldn't > >>>> > > > because of > >>>> > > > having > >>>> > > > to do everything in C. And it would be interesting to carry out > >>>> > > > it out > >>>> > > > in > >>>> > > > FreeBSD as well since it has a larger community than NetBSD. > >>>> > > > > >>>> > > > I humbly request anyone who is interested in this project to be > >>>> > > > my > >>>> > > > potential mentor(s) for GSoC. > >>>> > > > > >>>> > > > More slides on kernel Lua in NetBSD - [9], [10]. > >>>> > > > > >>>> > > > Thanks, > >>>> > > > Saurav > >>>> > > > > >>>> > > > [1] - http://www.netbsd.org/~lneto/dls14.pdf > >>>> > > > [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf > >>>> > > > [3] - https://github.com/jsonn/src/tree/trunk/external/mit/lua/ > >>>> > > > dist/src > >>>> > > > [4] - > >>>> > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/module > >>>> > > > s/lua > >>>> > > > [5] - > >>>> > > > https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/module > >>>> > > > s/luasystm > >>>> > > > [6] - https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sbin > >>>> > > > /luactl > >>>> > > > [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetBSD-current > >>>> > > > [8] - http://www.lua.org/tests/ > >>>> > > > [9] - > >>>> > > > https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2012 > >>>> > > > /kernel_mode_lua.pdf > >>>> > > > [10] - https://www.lua.org/wshop13/Cormack.pdf > >>>> > > > _______________________________________________ > >>>> > > > freebsd-hackers@freebsd.org mailing list > >>>> > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >>>> > > > To unsubscribe, send any mail to > >>>> > > > "freebsd-hackers-unsubscribe@freebsd.org" > >>>> > > This may be quite a nice thing to have. Another upcoming use for > >>>> > > LUA in the kernel is ZFS Channel Programs. These allow a number > >>>> > > of ZFS operations to be completed as a single atomic transaction. > >>>> > > > >>>> > > I would hope we could structure this in such a way as to not end > >>>> > > up with two copies of Lua in the kernel. > >>>> > There's also a 3/4 finished lua in the boot loader that you might > >>>> > be > >>>> > able to leverage as well.... > >>>> I'd like to see that finished. While Devin has done Heroic work with > >>>> the forth in the loader, I think it's time has come. > >>>> It' be nice to have something a little less '60s. > >>>> > >>>> > > >>>> > > >>>> > Warner > >>> > >>> I was under the impression that the "lua in bootloader" work was > >>> basically done and just needed testing, which nobody has done. I think > >>> it's all sitting in the projects/lua-bootloader branch in svn. > >> > >> The branch compiles. Testing has been done, but there's some missing > >> bits. It basically kinda works for the average case, but more advanced > >> uses of the bootloader still have sharp pointy edges on them, the > >> extent of the pointy edges is unknown. At this point the rebasing of > >> the branch is non-trivial due to the merge conflicts that have crept > >> in. They don't look awful, but when I tried to use git to rebase to a > >> more modern FreeBSD, there were lots of stupid things. Maybe I'll try > >> again... > >> > >> Warner > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@ > freebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >