From owner-freebsd-hackers@freebsd.org Mon Mar 9 10:31:59 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DC41625E4D7; Mon, 9 Mar 2020 10:31:59 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48bZKB10vyz4PRw; Mon, 9 Mar 2020 10:31:57 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 420086EE67; Mon, 9 Mar 2020 11:31:47 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7t3jq27jDR-D; Mon, 9 Mar 2020 11:31:46 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 336CA6EE66; Mon, 9 Mar 2020 11:31:46 +0100 (CET) To: "freebsd-virtualization@freebsd.org" , FreeBSD Hackers , "ports@freebsd.org" From: Willem Jan Withagen Subject: [RFC] Adding a Rados block driver to bhyve Message-ID: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> Date: Mon, 9 Mar 2020 11:31:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0 Thunderbird/68.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Content-Language: nl X-Rspamd-Queue-Id: 48bZKB10vyz4PRw X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wjw@digiware.nl designates 2001:4cb8:90:ffff::3 as permitted sender) smtp.mailfrom=wjw@digiware.nl X-Spamd-Result: default: False [-5.38 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[digiware.nl]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-3.10)[ip: (-9.51), ipnet: 2001:4cb8::/29(-4.68), asn: 28878(-1.32), country: NL(0.03)]; NEURAL_HAM_MEDIUM(-0.99)[-0.986,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:28878, ipnet:2001:4cb8::/29, country:NL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Mar 2020 10:32:00 -0000 Hi all, And sorry for crosspoing three groups, but the answer can/could be a mix of things to do in these three areas. I have a prototype of bhyve running on Rados/Ceph working:     https://github.com/freebsd/freebsd/pull/426 But there are a few catches on how to get it in the FreeBSd sources... 1) Easiest would be to just compile it in with the code of the current bhyve.     That will require librados/librbd libraries...     Ceph of this purpose is LGPL2/3 and could go into contrib.     In this case bhyve will hold the rbd-driver by default and a user does not     need to do anything by himself     But I have the feeling that this is the most unwanted scenario 2) User first installs a Ceph package and FreeBSD sources, and then recompiles     bhyve with the option BHYVE_RBD.     And then reinstalls this new version as bhyve or bhyve-rbd in /usr/sbin 3) Create a bhyve-rbd port.     Problem with that is that it will require the FreeBSD source tree for the     bhyve sources, but there is no Ports option for that?     Or bhyve sources are manually copied into the port. And then     try to keep these sources up to date.     Then compile and install a bhyve-rbd into /usr/local/sbin 4) Create a bhyve-blockrbd port.     This is much like 3) but instead of building a bhyve-rbd executable,     it delivers a libblockrbd.so that is dynamically loadable by the     standaard bhyve that comes with base.     For this bhyve needs to be extended with dynamic loadable driver modules.     This is reasonably doable, but is this acceptable for the bhyve maintainers?     For building the port, the bhyve-blockrbd code will only need a limited set     of files from /usr/src/usr.bin/bhyve thus limiting the chance of running out     sequence with the bhyve from base. Looking over these 4 options, I think that 4 is the most desirable one? But 2 would parhaps be workable for users as well, but the project might think otherwise. Are there other options? And/or is 4 the best way to go, with 2 as a nice intermediate? Thanx, --WjW