From owner-freebsd-arm@freebsd.org Thu Jan 3 09:25:34 2019 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFBAA1422040 for ; Thu, 3 Jan 2019 09:25:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-10.consmr.mail.ne1.yahoo.com (sonic308-10.consmr.mail.ne1.yahoo.com [66.163.187.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD39B6BAF1 for ; Thu, 3 Jan 2019 09:25:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: A3.moPAVM1kRZpCBbKh8mEmWkGJygBBJGnS5Y6_9lxYMa5tsRMD9lLTEYaS5G9g QMmOIerjKI8cxpll4aQEOpzqRbFVy4hoZPDYlXhtfeOY4tQgxrhiF4Lrr3U8NoFQLQwxG6pd5pp3 FEDCgiLrd_itW47NZsxDm6p8lqM1IH.cwqE3TT64peQMN6FSEeJNtvRK1CJZf7bO2vc_g8yXbhFz crgOqmYpfV6lAQr3jO7HM2TJBpbY550cS9ex8ZDaxD7Rhs5AOJmXNnyg7Xl7q4vqa0NEJB9guF8Q f8GQQqsu1KmQrybYhezfPrK1jlFmUfE4rN_aT.t90Wd9yLaF1x5sPoUvHWCqZVbsGPQEyH1L.V6A .yUcRK8T6Ko9hEpV6Hgibupe4br26A0IsgiesR.Za.jhkOUnrCfUP0.tT6EwQNmlH8vALt0szP.p 1C4vTGqFYkmzCfbGxLe9f1S8ZcYk.jUO1QDYxpf8wG_wbXITBqaWYddYzy1FtPMDVivTEfvPRW8p JdPn9LS3_Vtg7MEIrsQjx4mBGp5x7gzNAHp1A2veo3PXJWbWOlQu3zETHzNGBmV5WRuSwqIyMf3j adjDzOmdMeZIZ8cPWN_2yF0PRpbDfXket3k20Yd45l0pyJ1D3OzvKMf_OmMOB00HjHeemyPV6jpa MvjokmO4hrfOP6VMllk0jvhy8YBpH6H70D7yW61eF0Cin_SdEJ88g7zXpVeFGSiATpbOrTgSYLMC GxNaXSb9fWgnwQm72cX0Gw.mCVIcMfshVJe4Tms3YN50QMUL.nSxlQuwY7.GcUN3V7Tn9W0HUjUE gjfU3jNowTCg51gJIMnqD0gFlE4KHFaWAHjuFysksJcjTm_lWYuFjq5WEUQeUWgnTftYtIr1KL_v 4SHM1EvZ0AxZcpKVHhK8XDbmEuwEOV2pOCBEe98S_J9Gl6Fa278epFevje2ufJlc0wPAdaWzmC_z I.tcjYm3XbUtSZi.tGt1SiYLXijBSJbDo.Yoh3ccueZwbbFbjD6Jip7aDhWkq9wObvpM8QqZ89Lb dVZH_rVuts_neNipjSmR9hXE2qsTH7A7PXNNP.J5waLqfLgRVc0otlUh_LociA9Ke5TvsIo4vmDe hnsd0Fg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.ne1.yahoo.com with HTTP; Thu, 3 Jan 2019 09:25:27 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.109]) ([67.170.167.181]) by smtp402.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID b8f2f43454d6314b563298e05f0013d4; Thu, 03 Jan 2019 09:25:25 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Under qemu-aarch64-static "wc /dev/null" gets "Unsupported ancillary data: 1/0" from a sendmsg attempt: because of wrong cmsg_len type in target_cmsghdr Message-Id: <22184643-4320-4B7C-86DA-A71DF62D4543@yahoo.com> Date: Thu, 3 Jan 2019 01:25:23 -0800 Cc: freebsd-arm To: Kyle Evans , freebsd-emulation@freebsd.org X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: CD39B6BAF1 X-Spamd-Bar: + X-Spamd-Result: default: False [2.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.80)[0.801,0]; NEURAL_HAM_LONG(-0.01)[-0.012,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.22)[ip: (4.20), ipnet: 66.163.184.0/21(1.11), asn: 36646(0.88), country: US(-0.08)]; NEURAL_SPAM_MEDIUM(0.50)[0.497,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[33.187.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2019 09:25:35 -0000 [This note follows the investigation sequence, ending with the important conclusions.] My test context here is a poudriere-devel bulk -i for a amd64->aarch64 context. wc /dev/null or wc //dev/null does: # wc /dev/null Unsupported ancillary data: 1/0 that then hangs-up until I ^C to get back to a prompt. Here is what ktrace/kdump shows the process before the hang through when I hit ^C to stop the hang-up: . . . 98475 101033 qemu-aarch64-static 0.000340 CALL = sigprocmask[340](SIG_BLOCK,0x7ffffffe3c80,0x7ffffffe3d80) 98475 101033 qemu-aarch64-static 0.000003 RET sigprocmask[340] 0 98475 101033 qemu-aarch64-static 0.000001 CALL = pselect[522](0x6,0,0x7ffffffe3fb0,0,0,0x7ffffffe3d80) 98475 101033 qemu-aarch64-static 0.000001 RET pselect[522] 1 98475 101033 qemu-aarch64-static 0.000001 CALL = sigprocmask[340](SIG_SETMASK,0x7ffffffe3c80,0) 98475 101033 qemu-aarch64-static 0.000001 RET sigprocmask[340] 0 98475 101033 qemu-aarch64-static 0.000042 CALL = write[4](0x2,0x7ffffffe3480,0x20) 98475 101033 qemu-aarch64-static 0.000036 GIO fd 2 wrote 32 bytes "Unsupported ancillary data: 1/0 " 98475 101033 qemu-aarch64-static 0.000003 RET write[4] 32/0x20 98475 101033 qemu-aarch64-static 0.000001 CALL = sendmsg[28](0x5,0x7ffffffe3c28,0) 98475 101033 qemu-aarch64-static 0.000003 RET sendmsg[28] -1 errno 22 = Invalid argument 98475 101033 qemu-aarch64-static 0.000184 CALL close[6](0x3) 98475 101033 qemu-aarch64-static 0.000040 RET close[6] 0 98475 101033 qemu-aarch64-static 0.000017 CALL close[6](0x7) 98475 101033 qemu-aarch64-static 0.000005 RET close[6] 0 98475 101033 qemu-aarch64-static 0.000002 CALL = sigprocmask[340](SIG_BLOCK,0x7ffffffe3c80,0x7ffffffe3d80) 98475 101033 qemu-aarch64-static 0.000001 RET sigprocmask[340] 0 98475 101033 qemu-aarch64-static 0.000001 CALL = pselect[522](0x6,0x7ffffffe3dd0,0,0,0,0x7ffffffe3d80) 98475 101539 qemu-aarch64-static 0.000089 RET nanosleep[240] 0 98475 101539 qemu-aarch64-static 0.000042 CALL = _umtx_op[454](0x86101f008,UMTX_OP_WAIT_UINT_PRIVATE,0,0,0) 98475 101033 qemu-aarch64-static 15.845396 RET pselect[522] -1 errno = 4 Interrupted system call Note the qemu-aarch64 genrated message and the later: sendmsg[28] -1 errno 22 Invalid argument The qemu-*-static code that wrote the message is from t2h_freebsd_cmsg and is: if ((cmsg->cmsg_level =3D=3D TARGET_SOL_SOCKET) && (cmsg->cmsg_type =3D=3D SCM_RIGHTS)) { int *fd =3D (int *)data; int *target_fd =3D (int *)target_data; int i, numfds =3D len / sizeof(int); for (i =3D 0; i < numfds; i++) { fd[i] =3D tswap32(target_fd[i]); } } else if ((cmsg->cmsg_level =3D=3D TARGET_SOL_SOCKET) && (cmsg->cmsg_type =3D=3D SCM_TIMESTAMP) && (len =3D=3D sizeof(struct timeval))) { /* copy struct timeval to host */ struct timeval *tv =3D (struct timeval *)data; struct target_freebsd_timeval *target_tv =3D (struct target_freebsd_timeval *)target_data; __get_user(tv->tv_sec, &target_tv->tv_sec); __get_user(tv->tv_usec, &target_tv->tv_usec); } else { gemu_log("Unsupported ancillary data: %d/%d\n", cmsg->cmsg_level, cmsg->cmsg_type); memcpy(data, target_data, len); } =20 Well it turns out that qemu_*-static 's code has: struct target_cmsghdr { abi_long cmsg_len; int32_t cmsg_level; int32_t cmsg_type; }; where for amd64 target_cmsghdr has: (gdb) p/d sizeof(struct target_cmsghdr) $2 =3D 16 (gdb) p/d sizeof(((struct target_cmsghdr *)0)->cmsg_len)=20 $5 =3D 8 (gdb) p/d &((struct target_cmsghdr *)0)->cmsg_level $4 =3D 8 (gdb) p/d &((struct target_cmsghdr *)0)->cmsg_type=20 $1 =3D 12 which does not match the amd64 or aarch64 native: struct cmsghdr { socklen_t cmsg_len; /* data byte count, = including hdr */ int cmsg_level; /* originating protocol = */ int cmsg_type; /* protocol-specific = type */ /* followed by u_char cmsg_data[]; */ }; =20 because the cmsghdr's cmsg_len is smaller, even on a 64-bit = architecture: (gdb) p/d sizeof(((struct cmsghdr *)0)->cmsg_len) $6 =3D 4 /usr/include/arpa/inet.h:typedef __socklen_t socklen_t; /usr/include/netinet/in.h:typedef __socklen_t socklen_t; /usr/include/netinet6/in6.h:typedef __socklen_t socklen_t; /usr/include/sys/_types.h:typedef __uint32_t __socklen_t; /usr/include/sys/socket.h:typedef __socklen_t socklen_t; . . . /usr/include/netdb.h:typedef __socklen_t socklen_t; so abi_long does not match socklen_t for 64-bit architectures. So code such as in t2h_freebsd_cmsg: cmsg->cmsg_level =3D tswap32(target_cmsg->cmsg_level); cmsg->cmsg_type =3D tswap32(target_cmsg->cmsg_type); is not using the correct target offsets when aarch64 is the target that it is extracting from (for example). For comparison on a 64-bit architecture: (gdb) p/d sizeof(struct cmsghdr) $1 =3D 12 (gdb) p/d &((struct cmsghdr *)0)->cmsg_level $2 =3D 4 (gdb) p/d &((struct cmsghdr *)0)->cmsg_type=20 $3 =3D 8 I do not yet have a tested change. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)