Date: Tue, 10 Mar 2020 10:48:53 -0600 From: Alan Somers <asomers@freebsd.org> To: Willem Jan Withagen <wjw@digiware.nl> Cc: "freebsd-virtualization@freebsd.org" <virtualization@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: [RFC] Adding a Rados block driver to bhyve Message-ID: <CAOtMX2iYx7CAd2Axo_reeZopWYEd_i%2ByHk5NeKt4bt88ecZTAg@mail.gmail.com> In-Reply-To: <ba937caf-ce4b-eeb9-8915-f9b9910aa711@digiware.nl> References: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> <CAOtMX2iyhS230oCysYx3YC72B8TwFrrkcXtCoMCYK85KbFORaQ@mail.gmail.com> <c2090191-920b-3d2f-f797-75a448b18ad6@digiware.nl> <CAOtMX2iwJ1_VVDXOnSP6nui-JcYMwndXMex8fxtyi6qEikUX8A@mail.gmail.com> <936ed7c2-99d2-5df8-de3f-f64f28d2ba6f@digiware.nl> <CAOtMX2gKkh=Fik9wjHoAx9Gt9zwePMjmRaGP63eE2quNTmyDGw@mail.gmail.com> <ba937caf-ce4b-eeb9-8915-f9b9910aa711@digiware.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 10, 2020 at 10:28 AM Willem Jan Withagen <wjw@digiware.nl> wrote: > On 10-3-2020 17:21, Alan Somers wrote: > > On Tue, Mar 10, 2020 at 9:41 AM Willem Jan Withagen <wjw@digiware.nl> > wrote: > >> On 10-3-2020 16:15, Alan Somers wrote: >> >> On Tue, Mar 10, 2020 at 3:59 AM Willem Jan Withagen <wjw@digiware.nl> >> wrote: >> >>> On 9-3-2020 14:46, Alan Somers wrote: >>> >>> On Mon, Mar 9, 2020 at 4:32 AM Willem Jan Withagen <wjw@digiware.nl> >>> wrote: >>> >>>> Hi all, >>>> >>>> And sorry for crosspoing three groups, but the answer can/could be a mix >>>> of things to do in these three areas. >>>> >>>> I have a prototype of bhyve running on Rados/Ceph working: >>>> https://github.com/freebsd/freebsd/pull/426 >>>> >>>> .......... >>> >>> 4) Create a bhyve-blockrbd port. >>>> This is much like 3) but instead of building a bhyve-rbd >>>> executable, >>>> it delivers a libblockrbd.so that is dynamically loadable by the >>>> standaard bhyve that comes with base. >>>> >>> ............ >>> >>> > Great work! I also agree that option 4 sounds like the best. There's >>> precedent for ports that >>> > require the FreeBSD Sources. For example, see devel/py-libzfs or >>> emulators/virtualbox-ose. >>> > You just need to define the SRC_BASE variable. >>> Hi Alan, >>> >>> Thanx for the hint, and it made me check what is actually available >>> within the poudriere jail >>> And that does have full source, so the Makefile code is mainly for those >>> that build in a different way. >>> >>> I've got a proto version working when compiling stuff with `make >>> buildworld`, but run in the >>> problem that libblock_rbd.so is stripped in such a way that the symbol I >>> need is removed. >>> Using the unstripped version does work. >>> >>> Is there an incantation for the SRC Makefiles that builds a dynamical >>> loadable lib?? >>> And I'm still looking for a PORTS example of building a dynamical >>> loadable lib. >>> Or is there no generic code for that in the PORTS Mk files? >>> >>> --WjW >>> >>> BTW: Still haven't worked in your AIO code :( >>> >> >> There are plenty of dynamic libraries built with the SRC makefiles. For >> example, >> https://svnweb.freebsd.org/base/head/lib/libbsdstat/Makefile?view=markup >> . >> >> >> That looks dangerously close to what I have for libblock_rbd. >> === >> > cat Makefile-librbd >> # >> # $FreeBSD$ >> # >> >> PACKAGE=lib${LIB} >> >> .include <src.opts.mk> >> >> LIB= block_rbd >> SHLIB_MAJOR= 1 >> >> SRCS= block_rbd.c >> >> CFLAGS+=-I${SRCTOP}/sys >> CFLAGS+=-g -O0 -fPIC -rdynamic >> LDFLAGS+=-Wl,-export-dynamic,-Bdynamic >> CFLAGS+=-DWITHOUT_CAPSICUM >> >> LOCALBASE?= /usr/local >> CFLAGS+= -I${LOCALBASE}/include >> LDFLAGS+= -L${LOCALBASE}/lib -lrados -lrbd >> >> WARNS?= 2 >> >> === >> >> This is the code that mk.lib.bsd runs: >> objcopy --only-keep-debug libblock_rbd.so.1.full libblock_rbd.so.1.debug >> objcopy --strip-debug --add-gnu-debuglink=libblock_rbd.so.1.debug >> libblock_rbd.so.1.full libblock_rbd.so.1 >> >> So still I get a stripped lib in /usr/lib. And then the one and only >> symbol I need to load >> is not found. Copying libblock_rbd.so.1.full actually works for me. >> >> So either I'm doing it the wrong way, like special options on the symbols >> oid. >> Or mk.lib.bsd cannot deliver dlopen/dlsym-able files? >> >> And there are plenty of ports that build shared libraries too, just look >> at /usr/local/lib/*.so. However, the ports framework doesn't have much >> special code just to support building libraries. Instead the hard work is >> always done by the ports themselves. Some use autotools, some cmake, etc >> etc. The simplest port I can find that uses both SRC_BASE and INSTALL_LIB >> is this one: https://svnweb.freebsd.org/ports/head/devel/linux_libusb/ . >> >> Oke thanx, I'll have a look at it, and given that I can see most of the >> compile build stuff >> in the SRC_BASE version I'll get it to work. >> >> --WjW >> > > Try setting "STRIP= " in your makefile. That should prevent the > stripping. However, I think there's something wrong with your library, > too. The library should be usable even if it's stripped. > > I checked with objdump, and the symbol that I need is definitly not > present in the stripped version. > And it does not really matter if I declare it static or not. > But I'll give it a few more itterations to try it out. > Including 'STRIP= ' > > Thanx, > --WjW > What does "nm --dynamic libblock_rbd.so.1" show?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2iYx7CAd2Axo_reeZopWYEd_i%2ByHk5NeKt4bt88ecZTAg>