From owner-freebsd-arch@freebsd.org Thu Sep 28 19:06:18 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3EC24E0667C for ; Thu, 28 Sep 2017 19:06:18 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 124616A25D; Thu, 28 Sep 2017 19:06:18 +0000 (UTC) (envelope-from delphij@gmail.com) Received: by mail-it0-x231.google.com with SMTP id c195so2703926itb.4; Thu, 28 Sep 2017 12:06:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=HA4g94uwW/lagrKjNocCwobz0qggFyCfk5kRTYgbypA=; b=JDQ5tT4Z63Q5/Ko4AeFOa/7ipqy7j1qrbrS8PA9TsOBMGnlpL3i7FVTBUmiV3rJKUx W4AeX8fRASOS5+yyuaNj02l1359ndZqgBwzG5xNIJ8OuOjM7g5P8L8gINHkcpbvysVSO v7bedg6/dJtghXPXLEqCd+KmN8FFpWhAoWbjmdPf3La44xu95XYxVyEB4bkUjOqFG7q5 VMNNJngFCSEM7Q1fGHJUruwk34slQc/EoJUyw7CO631NIk/iI+3iJ91Buzm6/X8HalpK qmlPPosqcgIC6w8wCq6ykG7+h9I9VnjTRxGMYzqun7GerJsvSTeOFJI0j2+wP+ZsRNSG um2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=HA4g94uwW/lagrKjNocCwobz0qggFyCfk5kRTYgbypA=; b=Z2hcN2oAefKUsENibRlYtkis++wBm/XlkG+16h7XmSFsF6ZIfZZIKfm75JUWPvWSzD iFBKjF46/lndtzEpE6zlwxEXrI6yXE42xoPC5i3e/TzM31hwcIAF/rL//eUZ4f0XbfY1 8y43k4DN/WfJb6Au3oe9a35CxLzvfM3/pGg8xQ4VTivxS11SAQqDg4Gp3q6NCpdWo7OP 22hXzsZtX0lH2Q2EFigm35YEPevs7DTGgtvi/gw5HKoaQ8ir1NCmnyMGGhtM1w6JcAZL OXKKib7DFY3NNLEP9ga3Xnu3EVQYi/B1quVlbxNfFfwSwcHr2VWqpXiRACJnL+kaInfv YvsQ== X-Gm-Message-State: AMCzsaXw/XWu4uY1K/i92v+lkEW6oBjQ0iZSnqe3IgYcjfECz9rwIoEu 6GURWJvqC6b71ZrjMZls+3oFa3tHEpR6Np21CeDzyWUe X-Google-Smtp-Source: AOwi7QBVQoR8UqwyDSLOvTyYwmcxzsNYIi2zJNodK8Fg7tivCN/OI2n1GCnHFOixoccTyx6wundf0cY3h3Z69+Uk6SI= X-Received: by 10.36.167.5 with SMTP id a5mr3143903itf.69.1506625576610; Thu, 28 Sep 2017 12:06:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.12.74 with HTTP; Thu, 28 Sep 2017 12:06:15 -0700 (PDT) In-Reply-To: <13426777.FL7aYimd4u@ralph.baldwin.cx> References: <13426777.FL7aYimd4u@ralph.baldwin.cx> From: Xin LI Date: Thu, 28 Sep 2017 12:06:15 -0700 Message-ID: Subject: Re: lua in zfs To: John Baldwin Cc: freebsd-arch@freebsd.org, Ian Lepore , Andriy Gapon Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 19:06:18 -0000 On Thu, Sep 28, 2017 at 10:52 AM, John Baldwin wrote: > On Thursday, September 28, 2017 11:07:35 AM Warner Losh wrote: >> On Thu, Sep 28, 2017 at 10:50 AM, Andriy Gapon wrote: >> >> > On 28/09/2017 19:44, Andriy Gapon wrote: >> > > On 28/09/2017 18:55, Ian Lepore wrote: >> > >> Iirc, the big difference between 5.2.x and 5.3 is that the latter added >> > >> support for integers. It seems like that would be a good thing, in the >> > >> kernel. >> > > >> > > I am sure that the ZFS Lua represents numbers as integers and has no >> > floating >> > > point support at all. >> > >> > >> > Some more info: >> > https://github.com/illumos/illumos-gate/blob/master/usr/ >> > src/uts/common/fs/zfs/lua/README.zfs >> >> >> Reading this strongly suggests to me that we're going to have two versions >> of lua in the tree then: >> >> 1) This version is a modified version of 5.2.4 >> 2) Significant changes have been made to harden this interpreter, which >> won't be in 5.3.0 >> 3) A number of APIs have been removed, which makes it difficult to use as a >> general purpose thing. >> >> So I think that you should go ahead and import it. The level of work for my >> request is very high, and significantly diverges us from upstream. It's >> unfortunate that we'll have two versions. But this isn't really pure lua >> but a variant maybe better described as lua-zfs. > > Yes, that was my guess from Andriy's original e-mail. ZFS tends to be a bit > self-contained and includes its own version of zlib for example. In general > it seems to use its own version of 3rd party utils rather than sharing those > with other parts of the system. The zlib situation with ZFS is somewhat different, I think the goal was to share one copy of zlib in the whole Solaris kernel, and as such it was implemented as wrapper around regular zlib rather than a fork like in ZFS. The only reason it was imported this way, if I understood correctly, is that the zlib in FreeBSD kernel at the time was too old. Actually there was an attempt to unify zlib across the kernel a few years ago, by the way: https://svnweb.freebsd.org/base?view=revision&revision=199954 I'd be *really* happy to see we unify all zlib users into one, hopefully with the base userland contrib/zlib. Maybe a good hackathon goal? Similar situation applies to the LZ4 compression too, but the change in LZ4 is more intrusive than zlib: at the time it was added, there is no other consumers of LZ4, and there was no way to change memory allocator without making changes to LZ4 code itself (which was fixed by more recent LZ4 versions). If in the future we decided to have LZ4 in other FreeBSD kernel components, it would be beneficial to unify them too. The lua-zfs is different as many pointed out, and I think the benefit of importing the functionality is great enough to warrant having two copies of code at the same time, at least as an interim solution. Cheers,