Date: Sat, 2 Mar 2019 14:21:57 +0100 From: Willem Jan Withagen <wjw@digiware.nl> To: =?UTF-8?Q?T=c4=b3l_Coosemans?= <tijl@FreeBSD.org> Cc: FreeBSD Toolchain <freebsd-toolchain@freebsd.org>, Ed Maste <emaste@freebsd.org> Subject: Re: Linking problem with lld Message-ID: <cf912b3e-2699-0997-a0f6-4bb75fee4aab@digiware.nl> In-Reply-To: <29a75c78-f2df-e80b-d013-631584e0fc71@digiware.nl> References: <71079fbc-c3c0-9b93-0ae4-8ceda5d1f751@digiware.nl> <20190222223456.5d7840b7@kalimero.tijl.coosemans.org> <29a75c78-f2df-e80b-d013-631584e0fc71@digiware.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On 22-2-2019 23:47, Willem Jan Withagen wrote: > On 22-2-2019 22:34, Tijl Coosemans wrote: >> On Fri, 22 Feb 2019 16:09:09 +0100 Willem Jan Withagen <wjw@digiware.nl> >> wrote: >>> Hi Ed ea, >>> >>> In het ceph project, they started to use library versioning, so it >>> seems. >>> >>> But compiling/linking that now with Clang and lld I run into trouble >>> building the ceph-tools. >>> >>> This is what I get back of one of the Devs: >>> -------------- >>> My guess is that your linker doesn't support the new symbol versioning >>> exports and since the symbols are hidden by default they aren't visible >>> in the shared library. Previously there was a bug (since Luminous and >>> the switch the cmake) where every public and private symbol was >>> exported >>> by librados. >>> >>> We just need to know the magic words to use for your compiler/linker to >>> change the symbol visibility to public for the API methods. >>> >>> -------------- >>> >>> Now I looked thru the options on both Clang and lld, but nothing >>> obviously springs into focus, as to add to change.... >>> >>> This is how librados is build: >>> /usr/bin/c++ -fPIC -Wall -Wtype-limits -Wignored-qualifiers >>> -Winit-self -Wpointer-arith -Werror=format-security >>> -fno-strict-aliasing >>> -fsigned-char -Wno-unknown-pragmas -Wno-unused-function >>> -Wno-unused-local-typedef -Wno-varargs -Wno-gnu-designator >>> -Wno-missing-braces -Wno-parentheses -Wno-deprecated-register >>> -ftemplate-depth-1024 -Wnon-virtual-dtor -Wno-unknown-pragmas >>> -Wno-ignored-qualifiers -Wno-inconsistent-missing-override >>> -Wno-mismatched-tags -Wno-unused-private-field >>> -Wno-address-of-packed-member -DCEPH_DEBUG_MUTEX >>> -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc >>> -fno-builtin-realloc -fno-builtin-free -O0 -g >>> -Wno-unused-command-line-argument -fuse-ld=/usr/bin/ld.lld >>> -Wl,--exclude-libs,ALL >>> -Wl,--version-script=/home/jenkins/workspace/ceph-master/src/librados/librados.map >>> >> Can you provide this librados.map file? > https://github.com/ceph/ceph/blob/master/src/librados/librados.map: > ------ > LIBRADOS_PRIVATE { > local: *; > }; > > LIBRADOS_14.2.0 { > global: > extern "C++" { > ceph::buffer::v14_2_0::*; > librados::v14_2_0::*; > > "typeinfo for > librados::v14_2_0::ObjectOperation"; > "typeinfo name for > librados::v14_2_0::ObjectOperation"; > "vtable for librados::v14_2_0::ObjectOperation"; > > "typeinfo for > librados::v14_2_0::ObjectReadOperation"; > "typeinfo name for > librados::v14_2_0::ObjectReadOperation"; > "vtable for > librados::v14_2_0::ObjectReadOperation"; > > "typeinfo for > librados::v14_2_0::ObjectWriteOperation"; > "typeinfo name for > librados::v14_2_0::ObjectWriteOperation"; > "vtable for > librados::v14_2_0::ObjectWriteOperation"; > > "typeinfo for librados::v14_2_0::WatchCtx"; > "typeinfo name for librados::v14_2_0::WatchCtx"; > "vtable for librados::v14_2_0::WatchCtx"; > > "typeinfo for librados::v14_2_0::WatchCtx2"; > "typeinfo name for librados::v14_2_0::WatchCtx2"; > "vtable for librados::v14_2_0::WatchCtx2"; > }; > } LIBRADOS_PRIVATE; > ------ > --WjW > Still haven't been able to work this out, I know a bit more about versioning but I'n not sure that the way selected in Ceph is working for FreeBSD. I've collected most of the essentials below I hope. --WjW So the test program is simple enough: ==== #include "include/rados/librados.h" #include "gtest/gtest.h" TEST(Librados, CreateShutdown) { rados_t cluster; int err; err = rados_create(&cluster, "someid"); EXPECT_EQ(err, 0); rados_shutdown(cluster); } ==== librados is big, but in summary the releavant parts are: ==== #ifdef __cplusplus extern "C" { #endif ..... #define CEPH_RADOS_API ..... CEPH_RADOS_API int rados_create(rados_t *cluster, const char * const id); CEPH_RADOS_API void rados_shutdown(rados_t cluster); #ifdef __cplusplus } #endif ==== And in the implementation of librados in `librados_c.cc` it looks like: ==== .... #define LIBRADOS_C_API_BASE(fn) \ asm(".symver _" #fn "_base, " #fn "@") #define LIBRADOS_C_API_BASE_DEFAULT(fn) \ asm(".symver _" #fn ", " #fn "@@") #define LIBRADOS_C_API_DEFAULT(fn, ver) \ asm(".symver _" #fn ", " #fn "@@LIBRADOS_" #ver) extern "C" int _rados_create(rados_t *pcluster, const char * const id) { ....... } LIBRADOS_C_API_BASE_DEFAULT(rados_create); extern "C" void _rados_shutdown(rados_t cluster) { ....... } LIBRADOS_C_API_BASE_DEFAULT(rados_shutdown); ==== Then this is the Cmake receipe to bolt it all together into librados.so.2.0.0: ==== # C/C++ API add_library(librados ${CEPH_SHARED} librados_c.cc librados_cxx.cc $<TARGET_OBJECTS:common_buffer_obj>) if(ENABLE_SHARED) set_target_properties(librados PROPERTIES OUTPUT_NAME rados VERSION 2.0.0 SOVERSION 2 VISIBILITY_INLINES_HIDDEN ON) if(NOT APPLE) set_property(TARGET librados APPEND_STRING PROPERTY LINK_FLAGS " -Wl,--exclude-libs,ALL") set_property(TARGET librados APPEND_STRING PROPERTY LINK_FLAGS " -Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/librados.map") endif() if(WITH_STATIC_LIBSTDCXX) set_property(TARGET librados APPEND_STRING PROPERTY LINK_FLAGS " -static-libstdc++ -static-libgcc") endif() endif() target_link_libraries(librados PRIVATE librados_impl osdc ceph-common cls_lock_client ${BLKID_LIBRARIES} ${CRYPTO_LIBS} ${EXTRALIBS} ${GSSAPI_LIBRARIES}) install(TARGETS librados DESTINATION ${CMAKE_INSTALL_LIBDIR}) ===== Now if I look in librados.so for rados_create that gives: ==== > objdump -t librados.so.2.0.0 | grep rados_create 000000000008a430 l F .text 00000000000000a6 _ZL16rados_create_cctPKcP18CephInitParameters 000000000008a250 l F .text 00000000000001df _rados_create 000000000008a4e0 l F .text 000000000000028d _rados_create2 000000000009f490 l F .text 00000000000000a4 _rados_create_read_op 000000000008a770 l F .text 0000000000000085 _rados_create_with_context 000000000009dab0 l F .text 00000000000000a4 _rados_create_write_op 000000000008a4e0 l F .text 000000000000028d rados_create2 000000000008a250 l F .text 00000000000001df rados_create 000000000009f490 l F .text 00000000000000a4 rados_create_read_op 000000000008a770 l F .text 0000000000000085 rados_create_with_context 000000000009dab0 l F .text 00000000000000a4 rados_create_write_op ==== So now I'm sort of baffled why linking does not work: ==== /usr/bin/c++ -Wall -Wtype-limits -Wignored-qualifiers -Winit-self -Wpointer-arith -Werror=format-security -fno-strict-aliasing -fsigned-char -Wno-unknown-pragmas -Wno-unused-function -Wno-unused-local-typedef -Wno-varargs -Wno-gnu-designator -Wno-missing-braces -Wno-parentheses -Wno-deprecated-register -ftemplate-depth-1024 -Wnon-virtual-dtor -Wno-unknown-pragmas -Wno-ignored-qualifiers -Wno-inconsistent-missing-override -Wno-mismatched-tags -Wno-unused-private-field -Wno-address-of-packed-member -DCEPH_DEBUG_MUTEX -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -O0 -g -Wno-unused-command-line-argument -Wl,--export-dynamic CMakeFiles/unittest_librados.dir/librados.cc.o -o ../../../bin/unittest_librados -L/usr/local/lib -Wl,-rpath,/usr/local/lib:/home/jenkins/workspace/ceph-master/build/lib ../../../lib/libgmock_main.a ../../../lib/libgmock.a ../../../lib/libgtest.a -lpthread /usr/local/lib/libldap.so /usr/local/lib/liblber.so ../../../lib/librados.so.2.0.0 /usr/local/lib/libldap.so /usr/local/lib/liblber.so ld: error: undefined symbol: rados_create >>> referenced by librados.cc:9 (/home/jenkins/workspace/ceph-master/src/test/librados/librados.cc:9) >>> CMakeFiles/unittest_librados.dir/librados.cc.o:(Librados_CreateShutdown_Test::TestBody()) ld: error: undefined symbol: rados_shutdown >>> referenced by librados.cc:12 (/home/jenkins/workspace/ceph-master/src/test/librados/librados.cc:12) >>> CMakeFiles/unittest_librados.dir/librados.cc.o:(Librados_CreateShutdown_Test::TestBody()) c++: error: linker command failed with exit code 1 (use -v to see invocation) ====
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?cf912b3e-2699-0997-a0f6-4bb75fee4aab>