From owner-freebsd-virtualization@freebsd.org Fri Mar 20 23:45:50 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C9AB2276667 for ; Fri, 20 Mar 2020 23:45:50 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48kgQ55lrkz4R7B for ; Fri, 20 Mar 2020 23:45:49 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: by mailman.nyi.freebsd.org (Postfix) id 33CC3276666; Fri, 20 Mar 2020 23:45:49 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 338D6276665 for ; Fri, 20 Mar 2020 23:45:49 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [176.74.240.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48kgQ3206lz4R2n; Fri, 20 Mar 2020 23:45:46 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 61F78444CE; Sat, 21 Mar 2020 00:45:43 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V4zPCSKCW3gd; Sat, 21 Mar 2020 00:45:41 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 77927444CC; Sat, 21 Mar 2020 00:45:41 +0100 (CET) Subject: Re: [RFC] Adding a Rados block driver to bhyve To: cem@freebsd.org Cc: "freebsd-virtualization@freebsd.org" References: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> <936ed7c2-99d2-5df8-de3f-f64f28d2ba6f@digiware.nl> From: Willem Jan Withagen Message-ID: <2d86d513-efe0-d25f-ae20-0fae74afa1bc@digiware.nl> Date: Sat, 21 Mar 2020 00:45:41 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: nl X-Rspamd-Queue-Id: 48kgQ3206lz4R2n X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wjw@digiware.nl designates 176.74.240.9 as permitted sender) smtp.mailfrom=wjw@digiware.nl X-Spamd-Result: default: False [-5.70 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[digiware.nl]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-3.20)[ip: (-9.78), ipnet: 176.74.224.0/19(-4.89), asn: 28878(-1.38), country: NL(0.03)]; RCVD_IN_DNSWL_MED(-0.20)[9.240.74.176.list.dnswl.org : 127.0.9.2]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:28878, ipnet:176.74.224.0/19, country:NL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 23:45:50 -0000 On 10-3-2020 19:08, Conrad Meyer wrote: > On Tue, Mar 10, 2020 at 9:28 AM Willem Jan Withagen > wrote: >>>> problem that libblock_rbd.so is stripped in such a way that the >>>> symbol I need is removed. >>> So either I'm doing it the wrong way, like special options on the >>> symbols oid. >>> However, I think there's something wrong with your library, too. The >>> library should be usable even if it's stripped. > Yes, strip only removes local symbols (.symtab), not exported ones > (.dynsym). >> I checked with objdump, and the symbol that I need is definitly not >> present in the stripped version. > How are you defining the symbol intended for export? Is the symbol a > function or data? Does the compiler flag -fvisibility=hidden get used? > Which symbol is missing and what are the symptoms? >> And it does not really matter if I declare it static or not. > Not declaring it "static" is necessary, if not sufficient. Looking at > your code on github, here are some issues: * In block_if.h, you define > an object blocklocal_backend. This is a header, and every compilation > unit that pulls in the header will get its own copy of > blocklocal_backend. You probably want 'extern block_backend_t > blocklocal_backend;' instead. I have fixed this in a different way, so I do not need to play tricks with     blocklocal_backend to handle the legacy case.So that is removed. > * You SET_DECLARE block_backend_set in block_if.c, but I think it > needs to be in block_if.h. Looked at the Macro definition, and it is `extern` so it can go into block_if.h > * There is some weirdness around linker sets being removed by the > linker if they are empty, so you may want to add blockbackend_local to > the linker set in the main program. (It's unclear to me why > blockbackend_local is treated specially regardless.) However, I'm not > quite sure why DATA_SET() in block_rbd.c is not creating > __start_set_block_backend_set / __stop_set_block_backend_set exported > symbols. As Alan asked, please provide 'nm -D foo.so'. Best, Conrad It is dynamic loaded with dlopen()/dlsym at the moment. But are you suggesting that a `DATA_SET()` actually adds to the set during dynamic loading? Note that the __start_set_block_backend_set / __stop_set_block_backend_setis avialable in the library. --WjW