From owner-freebsd-current@freebsd.org Mon Dec 31 00:38:57 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89036143B898 for ; Mon, 31 Dec 2018 00:38:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-12.consmr.mail.bf2.yahoo.com (sonic304-12.consmr.mail.bf2.yahoo.com [74.6.128.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2F9F75565 for ; Mon, 31 Dec 2018 00:38:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 5ER293AVM1nYvxUVQp1SOJ2JY9oTqoEra0yamggZlr6CsuilPTlX3gomiy4MtUd Ai16CS9aKKe0y63Eu0yMAi4HgptjfXkUHxkx2xwotBJ81PkM35hsMX9WRbffkFH4wHX1XxgEkUVy qphqY1NniQt2sG0lE4VBG3wIFVVX._j9tv0B8bIQ7HX33PEnpqIi8kSSB8FCP6Duvw06QfkcPX90 j64kjbA60H1vx1oC2lPTu2XHILSDr6s0a9n1W8K_scpf7BNmSdgFPM4SZJ4Yl2X6iKguTX6i0qWX V6A0Mh8wfahwP7DA3eC1wXPmfPSmiEKfOdkk5ckpPaZJ30Qfgim9N9LHJt3IvmBx3cBiRara_7Ba LlxTiMTN.rrDZyXRzz.UgRMsG53XDvoGRKN1CITGni.GPFoKzPz25zK4zO_NUGDRIjcHgIlDyRXX GXK0e4HAKVM_.uHb50YfbZKmDp.I_vE.445lN16zMku9FkQ35gnzDUzJ5NemPhKjIxWtPSBCWwce ZKrErVCcrwLRwkoIcCnI.NG2eroXWJtwa9hsr_MgEhU2E7ue.wC5kg.pAddj5lA47_.DWtsaH0gk 6RGrWwsPmPvn_7_k7wSXy8P.8hJJkdhSHL9AjhNPjxd2iSWSN8H1tvTAGxGnpcNVxlgA6A8dwRW7 PWaVhGv6lAaNXGU_xxC68g2rvDZASL1M7mePK4xVOTtmycZflzCTJbWGQAaNvC2KqZCZ61rr2fdv U4w6kMiNrSMIf_6npPOMwv85gLKWY9yQelxlBs8lAq_9whoYozrqceWpsvzxW8crmcC_IyhQpPB6 XC_WL0uznyX9_8tEP_Y8WK7TGykFbWPbwoC0Pk8gI2gOCXFCn9Y7G2obXG1zw9J.sljgCfdmQH_u 4pf6A7HIm_gMJPM4TojLBoWqcYhGN4BOtK6wERh2944zfcQerFxrp5HZEAcF2z8m25LgZ4vqtlb9 3NIV9Ki8UcyKekwfOkzNGdVbnzHmC1_RROrbOrsTsPsLnwhQI_060DFpqseYJaYNkOlH50Dezy1x Q1HKf093xalJFKBw45_dU_i3.dOKz.iUlJ4HDvgTtUHA0PkZWyKz0TulI.U9OtjLxiQ0op.A- Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.bf2.yahoo.com with HTTP; Mon, 31 Dec 2018 00:38:49 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.109]) ([67.170.167.181]) by smtp422.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 57b6b0495726289929d5de26bdecfcdf; Mon, 31 Dec 2018 00:38:44 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: A reliable port cross-build failure (hangup) in my context (amd64->armv7 cross build, with native-tool speedup involved) [details of a specific qemu-arm-static source code problem] From: Mark Millard In-Reply-To: <2E3F6196-4652-40D2-937F-8860B6005A35@yahoo.com> Date: Sun, 30 Dec 2018 16:38:42 -0800 Cc: freebsd-emulation@freebsd.org, FreeBSD Current , ports-list freebsd , freebsd-arm , FreeBSD Toolchain Content-Transfer-Encoding: quoted-printable Message-Id: References: <865A13C8-9749-486E-9F79-5EEDDECBE621@yahoo.com> <0154C3AC-D85B-4FCF-BA63-454BC26BC1A2@yahoo.com> <13f5e4dd-33fb-2170-e31a-1b5d5f155869@freebsd.org> <2E3F6196-4652-40D2-937F-8860B6005A35@yahoo.com> To: mmel@freebsd.org X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: F2F9F75565 X-Spamd-Bar: / X-Spamd-Result: default: False [0.94 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.31)[-0.307,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.46)[0.457,0]; NEURAL_HAM_LONG(-0.11)[-0.108,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.41)[ip: (4.36), ipnet: 74.6.128.0/21(1.54), asn: 26101(1.23), country: US(-0.08)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[35.128.6.74.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2018 00:38:57 -0000 On 2018-Dec-28, at 12:12, Mark Millard wrote: > On 2018-Dec-28, at 05:13, Michal Meloun = wrote: >=20 >> Mark, >> this is known problem with qemu-user-static. >> Emulation of every single interruptible syscall is broken by design = (it >> have signal related races). Theses races cannot be solved without = major >> rewrite of syscall emulation code. >> Unfortunately, nobody actively works on this, I think. >>=20 >=20 > Thanks for the note setting some expectations. > . . . It turns out that I've been through (part of?) this before and mikael.urankar@gmail.com had back then provided a qemu-user-static patch (that might have been arm specific or 32-bit target specific when running on a 64-bit host). (The qemu-user-static code structure seems to have changed some afterwards and the patch is no longer where he had pointed me to back then.) To show size and offsets on armv7 vs. armd64 for struct kevent I use: # more kevent_size_offsets.c=20 #include "/usr/include/sys/event.h" // kevent #include // offsetof #include // printf int main() { printf("%lu\n", (unsigned long) sizeof(struct kevent)); printf("ident %lu\n", (unsigned long) offsetof(struct kevent, = ident)); printf("filter %lu\n", (unsigned long) offsetof(struct kevent, = filter)); printf("flags %lu\n", (unsigned long) offsetof(struct kevent, = flags)); printf("fflags %lu\n", (unsigned long) offsetof(struct kevent, = fflags)); printf("data %lu\n", (unsigned long) offsetof(struct kevent, = data)); printf("udata %lu\n", (unsigned long) offsetof(struct kevent, = udata)); printf("ext %lu\n", (unsigned long) offsetof(struct kevent, = ext)); return 0; } It ends up showing on armv7 (under qemu-arm-static insteead of native, = not that it matters here): # ./a.out 64 ident 0 filter 4 flags 6 fflags 8 data 16 udata 24 ext 32 On amd64 (native) it ends up as: # ./a.out 64 ident 0 filter 8 flags 10 fflags 12 data 16 udata 24 ext 32 Thus a translation of layout is required when hosted. This is for: struct kevent { __uintptr_t ident; /* identifier for this event */ short filter; /* filter for event */ unsigned short flags; /* action flags for kqueue */ unsigned int fflags; /* filter flag value */ __int64_t data; /* filter data value */ void *udata; /* opaque user data identifier = */ __uint64_t ext[4]; /* extensions */ }; But qemu-user-static has for translation purposes: struct target_freebsd_kevent { abi_ulong ident; int16_t filter; uint16_t flags; uint32_t fflags; int64_t data; abi_ulong udata; uint64_t ext[4]; } __packed; (note the __packed) for which in amd64's qemu_arm_static has the size and offsets: # gdb qemu-arm-static . . . (gdb) p/d sizeof(struct target_freebsd_kevent) $1 =3D 56 (gdb) p/d &((struct target_freebsd_kevent *)0)->ident $2 =3D 0 (gdb) p/d &((struct target_freebsd_kevent *)0)->filter $3 =3D 4 (gdb) p/d &((struct target_freebsd_kevent *)0)->flags $4 =3D 6 (gdb) p/d &((struct target_freebsd_kevent *)0)->fflags $5 =3D 8 (gdb) p/d &((struct target_freebsd_kevent *)0)->data $6 =3D 12 (gdb) p/d &((struct target_freebsd_kevent *)0)->udata $7 =3D 20 (gdb) p/d &((struct target_freebsd_kevent *)0)->ext $8 =3D 24 which which does not match the armv7 offsets for data, udata, or ext and does not have the right size for struct target_freebsd_kevent[] indexing to match armv7's struct target_freebsd_kevent[] indexing. This in turn makes the do_freebsd_kevent code do the wrong thing in its: struct target_freebsd_kevent *target_changelist, *target_eventlist; . . . for (i =3D 0; i < arg3; i++) { __get_user(changelist[i].ident, = &target_changelist[i].ident); __get_user(changelist[i].filter, = &target_changelist[i].filter); __get_user(changelist[i].flags, = &target_changelist[i].flags); __get_user(changelist[i].fflags, = &target_changelist[i].fflags); __get_user(changelist[i].data, &target_changelist[i].data); /* __get_user(changelist[i].udata, = &target_changelist[i].udata); */ #if TARGET_ABI_BITS =3D=3D 32 changelist[i].udata =3D (void = *)(uintptr_t)target_changelist[i].udata; tswap32s((uint32_t *)&changelist[i].udata); #else changelist[i].udata =3D (void = *)(uintptr_t)target_changelist[i].udata; tswap64s((uint64_t *)&changelist[i].udata); #endif __get_user(changelist[i].ext[0], = &target_changelist[i].ext[0]); __get_user(changelist[i].ext[1], = &target_changelist[i].ext[1]); __get_user(changelist[i].ext[2], = &target_changelist[i].ext[2]); __get_user(changelist[i].ext[3], = &target_changelist[i].ext[3]); } . . . for (i =3D 0; i < arg5; i++) { __put_user(eventlist[i].ident, &target_eventlist[i].ident); __put_user(eventlist[i].filter, = &target_eventlist[i].filter); __put_user(eventlist[i].flags, &target_eventlist[i].flags); __put_user(eventlist[i].fflags, = &target_eventlist[i].fflags); __put_user(eventlist[i].data, &target_eventlist[i].data); /* __put_user(eventlist[i].udata, = &target_eventlist[i].udata);*/ #if TARGET_ABI_BITS =3D=3D 32 tswap32s((uint32_t *)&eventlist[i].data); target_eventlist[i].data =3D (uintptr_t)eventlist[i].data; #else tswap64s((uint64_t *)&eventlist[i].data); target_eventlist[i].data =3D (uintptr_t)eventlist[i].data; #endif __put_user(eventlist[i].ext[0], = &target_eventlist[i].ext[0]); __put_user(eventlist[i].ext[1], = &target_eventlist[i].ext[1]); __put_user(eventlist[i].ext[2], = &target_eventlist[i].ext[2]); __put_user(eventlist[i].ext[3], = &target_eventlist[i].ext[3]); } I'll eventually do something to have struct target_freebsd_kevent for amd64-native targeting armv7 and see if that is sufficient to avoid the problem in my context. Previously removing the __packed was enough to make the structure the same size with the same offsets as for armv7. (Such might not have been appropriate to all targets.) armv6 would have the same problem as I understand things. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)