Date: Tue, 10 Mar 2020 17:28:10 +0100 From: Willem Jan Withagen <wjw@digiware.nl> To: Alan Somers <asomers@freebsd.org> Cc: "freebsd-virtualization@freebsd.org" <virtualization@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: [RFC] Adding a Rados block driver to bhyve Message-ID: <ba937caf-ce4b-eeb9-8915-f9b9910aa711@digiware.nl> In-Reply-To: <CAOtMX2gKkh=Fik9wjHoAx9Gt9zwePMjmRaGP63eE2quNTmyDGw@mail.gmail.com> References: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> <CAOtMX2iyhS230oCysYx3YC72B8TwFrrkcXtCoMCYK85KbFORaQ@mail.gmail.com> <c2090191-920b-3d2f-f797-75a448b18ad6@digiware.nl> <CAOtMX2iwJ1_VVDXOnSP6nui-JcYMwndXMex8fxtyi6qEikUX8A@mail.gmail.com> <936ed7c2-99d2-5df8-de3f-f64f28d2ba6f@digiware.nl> <CAOtMX2gKkh=Fik9wjHoAx9Gt9zwePMjmRaGP63eE2quNTmyDGw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10-3-2020 17:21, Alan Somers wrote: > On Tue, Mar 10, 2020 at 9:41 AM Willem Jan Withagen <wjw@digiware.nl > <mailto:wjw@digiware.nl>> wrote: > > On 10-3-2020 16:15, Alan Somers wrote: >> On Tue, Mar 10, 2020 at 3:59 AM Willem Jan Withagen >> <wjw@digiware.nl <mailto:wjw@digiware.nl>> wrote: >> >> On 9-3-2020 14:46, Alan Somers wrote: >>> On Mon, Mar 9, 2020 at 4:32 AM Willem Jan Withagen >>> <wjw@digiware.nl <mailto:wjw@digiware.nl>> wrote: >>> >>> Hi all, >>> >>> And sorry for crosspoing three groups, but the answer >>> can/could be a mix >>> of things to do in these three areas. >>> >>> I have a prototype of bhyve running on Rados/Ceph working: >>> https://github.com/freebsd/freebsd/pull/426 >>> >> .......... >>> >>> 4) Create a bhyve-blockrbd port. >>> This is much like 3) but instead of building a >>> bhyve-rbd executable, >>> it delivers a libblockrbd.so that is dynamically >>> loadable by the >>> standaard bhyve that comes with base. >>> >> ............ >> >> > Great work! I also agree that option 4 sounds like the >> best. There's precedent for ports that >> > require the FreeBSD Sources. For example, see >> devel/py-libzfs or emulators/virtualbox-ose. >> > You just need to define the SRC_BASE variable. >> Hi Alan, >> >> Thanx for the hint, and it made me check what is actually >> available within the poudriere jail >> And that does have full source, so the Makefile code is >> mainly for those that build in a different way. >> >> I've got a proto version working when compiling stuff with >> `make buildworld`, but run in the >> problem that libblock_rbd.so is stripped in such a way that >> the symbol I need is removed. >> Using the unstripped version does work. >> >> Is there an incantation for the SRC Makefiles that builds a >> dynamical loadable lib?? >> And I'm still looking for a PORTS example of building a >> dynamical loadable lib. >> Or is there no generic code for that in the PORTS Mk files? >> >> --WjW >> >> BTW: Still haven't worked in your AIO code :( >> >> >> There are plenty of dynamic libraries built with the SRC >> makefiles. For example, >> https://svnweb.freebsd.org/base/head/lib/libbsdstat/Makefile?view=markup >> . > > That looks dangerously close to what I have for libblock_rbd. > === > > cat Makefile-librbd > # > # $FreeBSD$ > # > > PACKAGE=lib${LIB} > > .include <src.opts.mk <http://src.opts.mk>>; > > LIB= block_rbd > SHLIB_MAJOR= 1 > > SRCS= block_rbd.c > > CFLAGS+=-I${SRCTOP}/sys > CFLAGS+=-g -O0 -fPIC -rdynamic > LDFLAGS+=-Wl,-export-dynamic,-Bdynamic > CFLAGS+=-DWITHOUT_CAPSICUM > > LOCALBASE?= /usr/local > CFLAGS+= -I${LOCALBASE}/include > LDFLAGS+= -L${LOCALBASE}/lib -lrados -lrbd > > WARNS?= 2 > > === > > This is the code that mk.lib.bsd runs: > objcopy --only-keep-debug libblock_rbd.so.1.full > libblock_rbd.so.1.debug > objcopy --strip-debug --add-gnu-debuglink=libblock_rbd.so.1.debug > libblock_rbd.so.1.full libblock_rbd.so.1 > > So still I get a stripped lib in /usr/lib. And then the one and > only symbol I need to load > is not found. Copying libblock_rbd.so.1.full actually works for me. > > So either I'm doing it the wrong way, like special options on the > symbols oid. > Or mk.lib.bsd cannot deliver dlopen/dlsym-able files? > > And there are plenty of ports that build shared libraries too, > just look at /usr/local/lib/*.so. However, the ports framework > doesn't have much special code just to support building > libraries. Instead the hard work is always done by the ports > themselves. Some use autotools, some cmake, etc etc. The simplest > port I can find that uses both SRC_BASE and INSTALL_LIB is this > one: https://svnweb.freebsd.org/ports/head/devel/linux_libusb/ . > > Oke thanx, I'll have a look at it, and given that I can see most > of the compile build stuff > in the SRC_BASE version I'll get it to work. > > --WjW > > > Try setting "STRIP= " in your makefile. That should prevent the > stripping. However, I think there's something wrong with your > library, too. The library should be usable even if it's stripped. I checked with objdump, and the symbol that I need is definitly not present in the stripped version. And it does not really matter if I declare it static or not. But I'll give it a few more itterations to try it out. Including 'STRIP= ' Thanx, --WjW
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ba937caf-ce4b-eeb9-8915-f9b9910aa711>