From owner-freebsd-emulation@freebsd.org Thu Jan 3 10:06:58 2019 Return-Path: Delivered-To: freebsd-emulation@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BF5E14234ED for ; Thu, 3 Jan 2019 10:06:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-3.consmr.mail.bf2.yahoo.com (sonic301-3.consmr.mail.bf2.yahoo.com [74.6.129.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 22D226DCF4 for ; Thu, 3 Jan 2019 10:06:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 7ood0GYVM1lSeOFmgC1s1hlM8tzFFMRcYIDJ7P3KPG0CMGHRpi32PWOcUt18enQ dClTYGeNWjl5NxLd.w6sK61tZVKenkajbbhvYmCKhcJw0LKxjdou_QQ6llHg5OpV1z1e8chmkV3A aal1TSp_QzvP6lfpklrk28Wn_Dxzwp6A_QYnq69O2xkNB2bJ3M5p8GMR3hQ1.oPfh3tHr8n8XLbW m6JcjEz9Uhp8IWaFJwcm7Hkv8wz8hyOcXFiMFbcQXKcY.m0QRNnnn.4W2DPKpa6UhzlS6gqV34oS XNQCwJ3yVzpxeHLDrP4XuoolJlIqzNRqIdORGk.0cNMTbNLCNKCNEkf7f7KHWdSv3NW0buz26c56 gxs_oLpdMf6Ons1OS_.O9joaAARO87nwYlx3wH5S05ew40FASH1rMvBuhf9d7fEnWCprTDmLh91y eYgdAZI7Ps9_6pk9zZi5SY5UhbnwrSE5qNxj1NEPranFw1KnSjY8WW6C3ZVOGw.9wAEt0tK7UxVx mWDhhLqGWm5WOetuu2pW62T0e7jK5PA1MFEQTMU0XWfWs3t8lY6rh.HQoIlxlx9ME4FOZoO6LW.q jqIGoREaXin4BlHgbBYEAvQ3vn9qKgAZqhRhdErM4t8tBU4nwi.ZJDdPtq9mPwLrm7b4IlarLzZO Voma1jUuTnHzvwKJIs0pxHNOoL699Qq1fKNUnF8nqH_RabkBOeqgUYY0LzQ8bZWj2Xa6CkGHhH0u joAiN.yj__HBqJN0gZERV4HI3ReBnFsd8TVueNKZcELjfm3GxkCr9Rn73huzBZw4RclHn9mAXzU3 Th0GY3hiK3mv6msSpCTHc0LiOIauwRDDI58eupm5vjiQQVBFA1O7XD_EApfr1AUblFdDcsJSACwG NaLh0hyeUs6KIaDmOW98oOxQV90woD3DBbplOCkYSPUTk7f7r000smD4.0R8G9rNwDBf_9pX9M12 UCja4DyJxNqS4tEbCsLJl2SYbsJB_NzoztV9_MkvcOUSCmMPEOzzIrRhzt203sl4ttt6L8hDqXq0 qu_oGR9qrPohzX77A8evXxTP6CsypwrfQcCgXyrpcYMLoe8wLMKr8LM1pcKQdSjMKfZ7uNzQ_Bu_ sU1dRcQ-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.bf2.yahoo.com with HTTP; Thu, 3 Jan 2019 10:06:55 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.109]) ([67.170.167.181]) by smtp425.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID e2e1ad36aec9e258eee1043557fc3499; Thu, 03 Jan 2019 10:06:53 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Under qemu-aarch64-static "wc /dev/null" gets "Unsupported ancillary data: 1/0" from a sendmsg attempt: because of wrong cmsg_len type in target_cmsghdr From: Mark Millard In-Reply-To: <22184643-4320-4B7C-86DA-A71DF62D4543@yahoo.com> Date: Thu, 3 Jan 2019 02:06:51 -0800 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: References: <22184643-4320-4B7C-86DA-A71DF62D4543@yahoo.com> To: Kyle Evans , freebsd-emulation@freebsd.org X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 22D226DCF4 X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.84 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.96)[0.957,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.43)[ip: (4.52), ipnet: 74.6.128.0/21(1.49), asn: 26101(1.19), country: US(-0.08)]; NEURAL_SPAM_MEDIUM(0.64)[0.643,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.32)[0.322,0]; RCVD_IN_DNSWL_NONE(0.00)[42.129.6.74.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-emulation@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development of Emulators of other operating systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2019 10:06:58 -0000 [Adding a for-reference note.] On 2019-Jan-3, at 01:25, Mark Millard wrote: > [This note follows the investigation sequence, > ending with the important conclusions.] >=20 > My test context here is a poudriere-devel bulk -i for a > amd64->aarch64 context. >=20 > wc /dev/null or wc //dev/null does: >=20 > # wc /dev/null > Unsupported ancillary data: 1/0 >=20 > that then hangs-up until I ^C to get back to a prompt. >=20 >=20 > Here is what ktrace/kdump shows the process before the hang through > when I hit ^C to stop the hang-up: >=20 > . . . > 98475 101033 qemu-aarch64-static 0.000340 CALL = sigprocmask[340](SIG_BLOCK,0x7ffffffe3c80,0x7ffffffe3d80) > 98475 101033 qemu-aarch64-static 0.000003 RET sigprocmask[340] 0 > 98475 101033 qemu-aarch64-static 0.000001 CALL = pselect[522](0x6,0,0x7ffffffe3fb0,0,0,0x7ffffffe3d80) > 98475 101033 qemu-aarch64-static 0.000001 RET pselect[522] 1 > 98475 101033 qemu-aarch64-static 0.000001 CALL = sigprocmask[340](SIG_SETMASK,0x7ffffffe3c80,0) > 98475 101033 qemu-aarch64-static 0.000001 RET sigprocmask[340] 0 > 98475 101033 qemu-aarch64-static 0.000042 CALL = write[4](0x2,0x7ffffffe3480,0x20) > 98475 101033 qemu-aarch64-static 0.000036 GIO fd 2 wrote 32 bytes > "Unsupported ancillary data: 1/0 > " > 98475 101033 qemu-aarch64-static 0.000003 RET write[4] 32/0x20 > 98475 101033 qemu-aarch64-static 0.000001 CALL = sendmsg[28](0x5,0x7ffffffe3c28,0) > 98475 101033 qemu-aarch64-static 0.000003 RET sendmsg[28] -1 errno = 22 Invalid argument > 98475 101033 qemu-aarch64-static 0.000184 CALL close[6](0x3) > 98475 101033 qemu-aarch64-static 0.000040 RET close[6] 0 > 98475 101033 qemu-aarch64-static 0.000017 CALL close[6](0x7) > 98475 101033 qemu-aarch64-static 0.000005 RET close[6] 0 > 98475 101033 qemu-aarch64-static 0.000002 CALL = sigprocmask[340](SIG_BLOCK,0x7ffffffe3c80,0x7ffffffe3d80) > 98475 101033 qemu-aarch64-static 0.000001 RET sigprocmask[340] 0 > 98475 101033 qemu-aarch64-static 0.000001 CALL = pselect[522](0x6,0x7ffffffe3dd0,0,0,0,0x7ffffffe3d80) > 98475 101539 qemu-aarch64-static 0.000089 RET nanosleep[240] 0 > 98475 101539 qemu-aarch64-static 0.000042 CALL = _umtx_op[454](0x86101f008,UMTX_OP_WAIT_UINT_PRIVATE,0,0,0) > 98475 101033 qemu-aarch64-static 15.845396 RET pselect[522] -1 errno = 4 Interrupted system call >=20 > Note the qemu-aarch64 genrated message and the later: > sendmsg[28] -1 errno 22 Invalid argument >=20 > The qemu-*-static code that wrote the message is from > t2h_freebsd_cmsg and is: >=20 > if ((cmsg->cmsg_level =3D=3D TARGET_SOL_SOCKET) && > (cmsg->cmsg_type =3D=3D SCM_RIGHTS)) { > int *fd =3D (int *)data; > int *target_fd =3D (int *)target_data; > int i, numfds =3D len / sizeof(int); >=20 > for (i =3D 0; i < numfds; i++) { > fd[i] =3D tswap32(target_fd[i]); > } > } else if ((cmsg->cmsg_level =3D=3D TARGET_SOL_SOCKET) && > (cmsg->cmsg_type =3D=3D SCM_TIMESTAMP) && > (len =3D=3D sizeof(struct timeval))) { > /* copy struct timeval to host */ > struct timeval *tv =3D (struct timeval *)data; > struct target_freebsd_timeval *target_tv =3D > (struct target_freebsd_timeval *)target_data; > __get_user(tv->tv_sec, &target_tv->tv_sec); > __get_user(tv->tv_usec, &target_tv->tv_usec); > } else { > gemu_log("Unsupported ancillary data: %d/%d\n", > cmsg->cmsg_level, cmsg->cmsg_type); > memcpy(data, target_data, len); > } >=20 > Well it turns out that qemu_*-static 's code has: >=20 > struct target_cmsghdr { > abi_long cmsg_len; > int32_t cmsg_level; > int32_t cmsg_type; > }; >=20 > where for amd64 target_cmsghdr has: >=20 > (gdb) p/d sizeof(struct target_cmsghdr) > $2 =3D 16 > (gdb) p/d sizeof(((struct target_cmsghdr *)0)->cmsg_len)=20 > $5 =3D 8 > (gdb) p/d &((struct target_cmsghdr *)0)->cmsg_level > $4 =3D 8 > (gdb) p/d &((struct target_cmsghdr *)0)->cmsg_type=20 > $1 =3D 12 >=20 > which does not match the amd64 or aarch64 native: >=20 > struct cmsghdr { > socklen_t cmsg_len; /* data byte count, = including hdr */ > int cmsg_level; /* originating protocol = */ > int cmsg_type; /* protocol-specific = type */ > /* followed by u_char cmsg_data[]; */ > }; =20 >=20 > because the cmsghdr's cmsg_len is smaller, even on a 64-bit = architecture: >=20 > (gdb) p/d sizeof(((struct cmsghdr *)0)->cmsg_len) > $6 =3D 4 >=20 > /usr/include/arpa/inet.h:typedef __socklen_t socklen_t; > /usr/include/netinet/in.h:typedef __socklen_t socklen_t; > /usr/include/netinet6/in6.h:typedef __socklen_t socklen_t; > /usr/include/sys/_types.h:typedef __uint32_t __socklen_t; > /usr/include/sys/socket.h:typedef __socklen_t socklen_t; > . . . > /usr/include/netdb.h:typedef __socklen_t socklen_t; >=20 > so abi_long does not match socklen_t for 64-bit architectures. >=20 > So code such as in t2h_freebsd_cmsg: >=20 > cmsg->cmsg_level =3D tswap32(target_cmsg->cmsg_level); > cmsg->cmsg_type =3D tswap32(target_cmsg->cmsg_type); >=20 > is not using the correct target offsets when aarch64 is the target > that it is extracting from (for example). >=20 > For comparison on a 64-bit architecture: >=20 > (gdb) p/d sizeof(struct cmsghdr) > $1 =3D 12 > (gdb) p/d &((struct cmsghdr *)0)->cmsg_level > $2 =3D 4 > (gdb) p/d &((struct cmsghdr *)0)->cmsg_type=20 > $3 =3D 8 >=20 >=20 > I do not yet have a tested change. >=20 On aarch64 (like on amd64): # more cmsghdr_size_offsets.c=20 #include "/usr/include/sys/socket.h" // cmsghdr #include // offsetof #include // printf int main() { printf("%lu\n", (unsigned long) sizeof(struct cmsghdr)); printf("cmsg_len %lu\n", (unsigned long) offsetof(struct = cmsghdr, cmsg_len)); printf("cmsg_level %lu\n", (unsigned long) offsetof(struct = cmsghdr, cmsg_level)); printf("cmsg_type %lu\n", (unsigned long) offsetof(struct = cmsghdr, cmsg_type)); return 0; } produces: # ./a.out 12 cmsg_len 0 cmsg_level 4 cmsg_type 8 which qemu-aarch64-static 's target_cmsghdr definitely does not match. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)