From nobody Mon Jan 17 22:31:09 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9A1D81964FE5 for ; Mon, 17 Jan 2022 22:31:20 +0000 (UTC) (envelope-from damian@dmcyk.xyz) Received: from mail-4323.proton.ch (mail-4323.proton.ch [185.70.43.23]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jd67q3DrXz4b58 for ; Mon, 17 Jan 2022 22:31:19 +0000 (UTC) (envelope-from damian@dmcyk.xyz) Date: Mon, 17 Jan 2022 22:31:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dmcyk.xyz; s=protonmail; t=1642458676; bh=1nsMQRyhSbCOMotn2ZmIrIQo/jyBV7SQbbFH52weeC4=; h=Date:To:From:Cc:Reply-To:Subject:Message-ID:In-Reply-To: References:From:To:Cc; b=A1ZdIr0ncEVJ9sPbcSlae42M58ale8UUXc2wFOxOVuquPUUL4ZGwveo29bNbWODt0 bCj8/d/2+SzhGtQZRmv7NdtxK2d1EYwTw/s7V6115Rx2rNocfxcmyFpo5uDIV/17a+ o7IzZ7xeIBlb38HrDr7uRgO7R6sq2Jrr6BXGcYFc= To: Konstantin Belousov From: Damian's Proton Mail Cc: "freebsd-hackers@freebsd.org" Reply-To: Damian's Proton Mail Subject: Re: amd64 syscall ABI (vs. Darwin) Message-ID: <94B30813-0034-4F90-9AAC-113402A1A3E8@dmcyk.xyz> In-Reply-To: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=1.3 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FROM_SUSPICIOUS_NTLD, PDS_OTHER_BAD_TLD shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4Jd67q3DrXz4b58 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=dmcyk.xyz header.s=protonmail header.b=A1ZdIr0n; dmarc=none; spf=pass (mx1.freebsd.org: domain of damian@dmcyk.xyz designates 185.70.43.23 as permitted sender) smtp.mailfrom=damian@dmcyk.xyz X-Spamd-Result: default: False [-1.03 / 15.00]; HAS_REPLYTO(0.00)[damian@dmcyk.xyz]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[dmcyk.xyz:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[185.70.43.23:from]; R_SPF_ALLOW(-0.20)[+ip4:185.70.43.0/24]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dmcyk.xyz]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[dmcyk.xyz:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(1.00)[1.000]; NEURAL_HAM_SHORT(-0.53)[-0.529]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-hackers]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N > On 17 Jan 2022, at 14:38, Konstantin Belousov wrote= : > > On Mon, Jan 17, 2022 at 12:41:59PM +0000, Damian Malarczyk wrote: >> Hello, >> >> I'm hacking on a toy project to run Darwin (MachO) binaries on FreeBSD. >> Currently I'm at a stage of syscalls support, and I've noticed a differe= nce in the amd64 ABI that I didn't expect. >> >> FreeBSD is changing values of some registers that aren't used as the sys= call output. e.g., r8-r11 are changed, while r12-r15 don't seem to be affec= ted. >> That's not the case on Darwin, from what I've seen onlyrax, rdx used as = syscall results are changed. >> It looks like FreeBSD's syscalls calling convention is more like standar= d function calling, and r8-r11 should be always caller saved. > It is not 'more like'. FreeBSD follows C ABI for amd64 for syscall > registers handling. An additional twist is that the registers which are > declared as calleee-clobered are zeroed to avoid kernel data leakage to > userspace. Oh I see, this explains it then. >> >> At a first glance Darwin approach seems more optimal, as less registers = get clobbered. Is there any specific reason why this isn't also the case on= FreeBSD? >> I'm also wondering where exactly the register values are changed. When I= look at thetrapframe contents in the sv_set_syscall_retvalsystem vector ca= llback the r8 register value is same as on the input, so it must be changed= somewhere later. Does anyone know where exactly this happens? > > Look at the sys/amd64/amd64/exceptions.S. The fast_syscall entry point > is where we receive control after the syscall instruction. A lot of new things in there for me, but the flow is clear. I was able to f= ind corresponding logic in XNU=E2=80=99s sources too. Earlier I said: > At a first glance Darwin approach seems more optimal But it=E2=80=99s instead the opposite/no difference at all, as in Darwin, t= hey explicitly restore/set all registers, including callee saved r12-r15. Explicitly preserving registers would prevent kernel data leakage too. Doin= g so in FreeBSD would also be an ABI compatible change I think, since users= shouldn=E2=80=99t rely on values in those registers. I=E2=80=99m curious if you see any obvious pros/cons with either approach, = or is it just a more arbitrary implementation choice? Not that I=E2=80=99d propose changing the ABI though, I also want my toy pr= oject to work as a plug-in kernel module. I guess the only other option to emulate Darwin's behaviour would be to int= ercept syscalls in userspace somehow first and manually preserve the regist= er values?