From owner-freebsd-current@freebsd.org Thu Mar 24 13:43:07 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEDC1ADBFF9 for ; Thu, 24 Mar 2016 13:43:07 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id A075315A0 for ; Thu, 24 Mar 2016 13:43:07 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9BFA7ADBFF6; Thu, 24 Mar 2016 13:43:07 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B788ADBFF5; Thu, 24 Mar 2016 13:43:07 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: from mail-oi0-x22f.google.com (mail-oi0-x22f.google.com [IPv6:2607:f8b0:4003:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F578159C; Thu, 24 Mar 2016 13:43:07 +0000 (UTC) (envelope-from damjan.jov@gmail.com) Received: by mail-oi0-x22f.google.com with SMTP id r187so61458497oih.3; Thu, 24 Mar 2016 06:43:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=fhbl+eNyCKGKIR5xBHUORGC+QeR2PY92Oq9G+q5nXV0=; b=MCbN7/Ntv9crj1TxWD3RmqdJpfU860oTfsV039zdo9a0EYN2BkvuAdb1johbXLz3ht RfOrFaUiv7Q3elvOK1mrOEMLvYCIyjIxvywlECH09b4Tn2lh2zwU5hik+7HzPtKVdgLr /5SlQcNdhckoyGHBajaUfPBMhKSyd89Im2xgItryzoYpGpapUbCeBjD7UhFz0qZbBkbV 1QWVo38DZrfHZtX2a7S//8+ydM1dIaYIfGgzgcWupsiO9ZonjEmmjXakIPKfMh9ktql0 8QZkSrFFuPUBhNm7sEfFadR9bourK0ae7/crJq//TsqNyOSsZJcLpmVmugCtgI+7nqFk n8Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=fhbl+eNyCKGKIR5xBHUORGC+QeR2PY92Oq9G+q5nXV0=; b=gUuX6jALXRCN9jKDKdg4LsrqmNlAdRcgM3NiU4wgsMDG3fUfW5UQRgyhS3TSnTEvoU EpYeCKQ2FQEehMLDY06Pq3NYKb0p5zZ6gKdNi0DE16rCWxcadHLm6XMUv6bgGnfYZB8u iimgQi3eRKto0vhjF/OLSFiqIFY+MM8APIOhlMw7zjCbCPQTDxFuasc1AL0sdqfTPc8K 0l5uJlz1RKzo2fA6afPiN1nbuPbwfWLIJJZuksiWG9EuFF/I24KVeXPvdB7CzkilkxXy JPB8Tulz/MR2Ysq/4eb4W56TVvW8zjd8/pxT22t3dchnn4GeDbyXn8CM2AZ4zrZ4+Y0s iuyQ== X-Gm-Message-State: AD7BkJLMlQYmwFLRGKFGTwKIWsuukq4SFAC1ZLnYAsSB3tIIYJedUnN+m7lEHKEOVPjUaWldWC5uiZhAfQXYeA== X-Received: by 10.202.57.133 with SMTP id g127mr4142000oia.120.1458826986439; Thu, 24 Mar 2016 06:43:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.38.168 with HTTP; Thu, 24 Mar 2016 06:42:46 -0700 (PDT) In-Reply-To: <7554521E-81AB-43DE-A7FC-A9F334F660B7@FreeBSD.org> References: <7554521E-81AB-43DE-A7FC-A9F334F660B7@FreeBSD.org> From: Damjan Jovanovic Date: Thu, 24 Mar 2016 15:42:46 +0200 Message-ID: Subject: Re: FreeBSD MachO File format, your comments on it. To: David Chisnall Cc: mokhi , emulation@freebsd.org, current@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Mar 2016 13:43:07 -0000 ELF itself is a disaster. Symbol lookup in ELF is process scoped, not library scoped like Windows's PE and Mac's Mach-O, so same named symbols from different libraries in the same process (loaded through any number of levels of indirection) can and do clash, resulting in memory corruption. This is why hacks like symbol versioning, RTLD_DEEPBIND on GNU's libc and -Bdirect on Solaris were invented. We suffer from this problem badly on FreeBSD, as Clang's C++ standard library and GCC's standard library don't have fully compatible ABIs, so when both are loaded into the same process and the incompatible C++ features are used -> memory corruption -> crash. Eg. compile Apache OpenOffice with GCC on a system built with Clang, and you'll see even the unit tests crash. This is why I am personally interested in alternatives like Mach-O. Damjan On Thu, Mar 24, 2016 at 12:51 PM, David Chisnall wro= te: > Hi, > > I=E2=80=99d slightly question the assertion that Mach-O is a well-designe= d format. For example, it has a hard limit of 16 section types, doesn=E2= =80=99t support COMDATs and so on. OS X uses a load of magic section names= to work around these limitations. > > Note that a Mach-O image activator is relatively easy, but a Mach-O rtld = is far more complex. It might be possible to port dyld from OS X, but as I= recall it depends quite heavily on the Mach kernel interfaces. > > On fat binaries, note that the support in the file format is pretty trivi= al. Far more support is needed in the image activator and rtld to determin= e the correct parts and map only them. If you=E2=80=99re interested in doi= ng this work, then I=E2=80=99d recommend looking at this proposed specifica= tion for fat ELF binaries: > > https://icculus.org/fatelf/ > > That said, I=E2=80=99m not totally convinced that fat binaries are actual= ly a good solution (unless you=E2=80=99re willing to go a step further than= Apple did and merge data sections) - NeXT managed very well shipping fat b= undles without using fat binaries and even had a special mode in ditto to s= trip out the foreign architectures when copying a bundle from a network sha= re to a local filesystem. > > Persuading clang to emit FreeBSD Mach-O binaries is probably harder than = you think. It=E2=80=99s quite easy to persuade it that Mach is a valid fil= e format for FreeBSD, but there are a *lot* of places where people conflate= =E2=80=98is Mach=E2=80=99 with =E2=80=98is Darwin=E2=80=99 in the Clang an= d LLVM sources. Finding all of these and making sure that they=E2=80=99re = really checking the correct one is difficult. > > Emulating OS X binaries may be interesting. NetBSD had a Mach / XNU comp= at layer for a while. The problem here is that the graphics stack interfac= es on OS X are completely different from any other *NIX system (as are the = kernel interfaces for sound), so the most that they could do was run comman= d-line and X11 Mac apps - not especially useful. Actually emulating OS X a= pps will need far more than that - OS X ships with about 500MB of framework= s, many of which are used by most applications. The GNUstep project is und= ermanned and hasn=E2=80=99t been able to keep up with the changes to the co= re Foundation and AppKit frameworks, let alone the rest. > > David > >> On 24 Mar 2016, at 09:13, mokhi wrote: >> >> Hi guys. >> I'm Mahdi Mokhtari (aka Mokhi between FreeBSD friends). >> >> I am working on adding Mach-O binary format to supported formats for Fre= eBSD. >> Not for emulations on first step, but as a native supported format >> just like a.out [or Elf] >> (though it can go in both ways too). >> >> There are good reasons to have Mach-O format support IMO. >> It's well/clear designed file format. >> Can supports multiple Arch by default (It's Fat Format). >> Because of its Fat Format support, it can even help porting/packaging ea= sier for >> projects such as Freebsd-arm or others IMO :D. >> At end (even not among its interesting parts, maybe :D) point, it >> leads and helps to have >> OSX emulation support on FreeBSD. >> >> BTW, i've coded[1] Mach-O support for FreeBSD with helps of >> FreeBSD-ppl on IRC about various aspects of this works (from >> fundamental points of VM-MAP, to SysEntVec for Mach-O format) and >> with help of Elf and a.out format codes and mach-o references. >> It's in Alpha state (or before it) IMO, as I'm not sure about some of >> its parts, but I've tested a mach-o formatted binary with it and it at >> least loads and maps it segments correctly :D. (it was actually a >> simple "return 0" C Code, compiled in a OSX, if you know how can I >> force my FreeBSD clang to produce mach-o files instead of ELF I'd be >> happy to know it, and I appreciate :D) >> >> I'd like to have your helps and comments on it, in hope to make it bette= r >> and make it ready for review. >> >> Thanks and thousands of regards, Mokhi. >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D >> [1] https://github.com/m0khi/FreeBSD_MachO >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or= g" > > _______________________________________________ > freebsd-emulation@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-emulation > To unsubscribe, send any mail to "freebsd-emulation-unsubscribe@freebsd.o= rg"