From owner-freebsd-hackers@freebsd.org Mon Mar 9 13:46:18 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 103AA262EDF; Mon, 9 Mar 2020 13:46:18 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f65.google.com (mail-ot1-f65.google.com [209.85.210.65]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48bfdM0W9Xz44k6; Mon, 9 Mar 2020 13:46:14 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f65.google.com with SMTP id i14so9582708otp.5; Mon, 09 Mar 2020 06:46:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dm9mCCHHaGSgYroZ1H/GGnoTLUEgtPx5U61kNf019Ms=; b=t0XLEiGNh+uuB9T83BsRSmhBRFCPMJK4Awa79sPTuPe40WUCMQHFNbSgc9UKgl0LPo zT38Na41egcavQUsbRguDiTaK06fvw/ajhVsasaCAR0zCzuca4DLceofNCOEcZ5UFcxL hrOtm6A/hUoYGg5srmKUT62vj5Tiyz0QOpUmfR3DqJjSjz89yuBEsbA8EqOPp6/XITZQ UKha3FIrdovT6LwWIPBCYIJFvKpGjTz44X1rbQ/J6Hh3teVlQXM20tuYfbI3dRtPFpQ7 TnFpFSfs7Vhwu/B0s/1ytlkKJz/s4V/QI0rf+/W8JauxOfft8tnh2wg6+UeXAoMfRXdw 24Yg== X-Gm-Message-State: ANhLgQ1eTAD39FbxB3153q1OuFANkDvSYTJalNeqGm6zilTCapdc8szw lZQ/2AzossWp8Vsens7W8tfB/B1e9Xh6xazWolWiDbn8Xb0= X-Google-Smtp-Source: ADFU+vuDOwFW13eFluwQKALgOYOuWAS2TfuIb0MaJR2cxng+ep0qOpFuLfGxAM8iR59eBCAv0FhSt3ym/3ndWuPUHfs= X-Received: by 2002:a05:6830:1182:: with SMTP id u2mr1353949otq.222.1583761573197; Mon, 09 Mar 2020 06:46:13 -0700 (PDT) MIME-Version: 1.0 References: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> In-Reply-To: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> From: Alan Somers Date: Mon, 9 Mar 2020 07:46:02 -0600 Message-ID: Subject: Re: [RFC] Adding a Rados block driver to bhyve To: Willem Jan Withagen Cc: "freebsd-virtualization@freebsd.org" , FreeBSD Hackers , "ports@freebsd.org" X-Rspamd-Queue-Id: 48bfdM0W9Xz44k6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.65 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.99 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[65.210.85.209.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RCPT_COUNT_THREE(0.00)[4]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[65.210.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-0.99)[ip: (-0.28), ipnet: 209.85.128.0/17(-2.98), asn: 15169(-1.65), country: US(-0.05)]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Mar 2020 13:46:18 -0000 On Mon, Mar 9, 2020 at 4:32 AM Willem Jan Withagen wrote: > Hi all, > > And sorry for crosspoing three groups, but the answer can/could be a mix > of things to do in these three areas. > > I have a prototype of bhyve running on Rados/Ceph working: > https://github.com/freebsd/freebsd/pull/426 > > But there are a few catches on how to get it in the FreeBSd sources... > > 1) Easiest would be to just compile it in with the code of the current > bhyve. > That will require librados/librbd libraries... > Ceph of this purpose is LGPL2/3 and could go into contrib. > In this case bhyve will hold the rbd-driver by default and a user > does not > need to do anything by himself > But I have the feeling that this is the most unwanted scenario > > 2) User first installs a Ceph package and FreeBSD sources, and then > recompiles > bhyve with the option BHYVE_RBD. > And then reinstalls this new version as bhyve or bhyve-rbd in > /usr/sbin > > 3) Create a bhyve-rbd port. > Problem with that is that it will require the FreeBSD source tree > for the > bhyve sources, but there is no Ports option for that? > Or bhyve sources are manually copied into the port. And then > try to keep these sources up to date. > Then compile and install a bhyve-rbd into /usr/local/sbin > > 4) Create a bhyve-blockrbd port. > This is much like 3) but instead of building a bhyve-rbd executable, > it delivers a libblockrbd.so that is dynamically loadable by the > standaard bhyve that comes with base. > > For this bhyve needs to be extended with dynamic loadable driver > modules. > This is reasonably doable, but is this acceptable for the bhyve > maintainers? > > For building the port, the bhyve-blockrbd code will only need a > limited set > of files from /usr/src/usr.bin/bhyve thus limiting the chance of > running out > sequence with the bhyve from base. > > Looking over these 4 options, I think that 4 is the most desirable one? > But 2 would parhaps be workable for users as well, but the project might > think > otherwise. > > Are there other options? > And/or is 4 the best way to go, with 2 as a nice intermediate? > > Thanx, > --WjW > Great work! I also agree that option 4 sounds like the best. There's precedent for ports that require the FreeBSD Sources. For example, see devel/py-libzfs or emulators/virtualbox-ose. You just need to define the SRC_BASE variable.