Date: Fri, 29 Dec 2023 21:12:35 +0100 From: Harry Schmalzbauer <freebsd@omnilan.de> To: Dimitry Andric <dim@FreeBSD.org> Cc: FreeBSD-STABLE <freebsd-stable@freebsd.org> Subject: Re: SIGILL when CPUTYPE set to anyting witjh avx and CFLAGS -O2 Message-ID: <62cc8fed-6acd-45fb-a138-ac7cd218191a@omnilan.de> In-Reply-To: <72107B4B-F279-471B-8A8F-5B94C5EEDA47@FreeBSD.org> References: <accdf012-1b04-4227-9b0c-a78e94133fc2@omnilan.de> <72107B4B-F279-471B-8A8F-5B94C5EEDA47@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/29/23 18:00, Dimitry Andric wrote:
> On 29 Dec 2023, at 11:14, Harry Schmalzbauer <freebsd@omnilan.de> wrote:
>>
>> at least since 14-release, I can't compile base/ports with CPUTYPE post nehalem. Any '-march' declaring AVX(*) capability leads to SIGILLing binaries with default -O2 CFLAGS.
...
>>
>> If I compile without CFLAGS -O2 ("CFLAGS= -pipe"), -march=skylake-avx512 the binary doens't crash with SIGILL.
>>
>> Is CPUTYPE supposed to stay untouched these days?
>
> Obviously not, but are you sure your target CPU is correct? If it is not, the compiler may insert unsupported instructions.
Ho Dimitry, thanks for your attention!
I have to admit that I know much too less about contemporary compilers
and CPUs likewise, but target CPU matches -march ;-)
I nailed it down to AVX instructions. Starting with 'sandybridge',
Intel CPUs provide avx instruction units, and starting with
CPUTYPE?=sandybridge, binaries crash on stable/14 (and late 14-current)
on the corresponding machines. But only in combination with CFLAGS -O2!
> In any case, for the above gdb session, try the command "disassemble" to see which particular instruction it crashed on. That may give more information.
Thanks for the hint!
This came out:
Program terminated with signal SIGILL, Illegal instruction.
Privileged opcode.
#0 0x000000000041e040 in init_yank ()
(gdb) disassemble
Dump of assembler code for function init_yank:
0x000000000041e030 <+0>: push %rbp
0x000000000041e031 <+1>: mov %rsp,%rbp
0x000000000041e034 <+4>: vpxor %xmm0,%xmm0,%xmm0
0x000000000041e038 <+8>: vmovdqa -0x202c10(%rip),%xmm1 # 0x21b430
=> 0x000000000041e040 <+16>: kxnorw %k0,%k0,%k1
0x000000000041e044 <+20>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e04f <+31>: vmovdqa -0x202b77(%rip),%xmm1 #
0x21b4e0
0x000000000041e057 <+39>: kxnorw %k0,%k0,%k1
0x000000000041e05b <+43>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e066 <+54>: vmovdqa -0x202c5e(%rip),%xmm1 #
0x21b410
0x000000000041e06e <+62>: kxnorw %k0,%k0,%k1
0x000000000041e072 <+66>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e07d <+77>: vmovdqa -0x202cf5(%rip),%xmm1 #
0x21b390
0x000000000041e085 <+85>: kxnorw %k0,%k0,%k1
0x000000000041e089 <+89>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e094 <+100>: vmovdqa -0x202b5c(%rip),%xmm1 #
0x21b540
0x000000000041e09c <+108>: kxnorw %k0,%k0,%k1
0x000000000041e0a0 <+112>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e0ab <+123>: vmovdqa -0x202e43(%rip),%xmm1 #
0x21b270
0x000000000041e0b3 <+131>: kxnorw %k0,%k0,%k1
0x000000000041e0b7 <+135>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e0c2 <+146>: vmovdqa -0x202d0a(%rip),%xmm1 #
0x21b3c0
0x000000000041e0ca <+154>: kxnorw %k0,%k0,%k1
0x000000000041e0ce <+158>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e0d9 <+169>: vmovdqa -0x202ac1(%rip),%xmm1 #
0x21b620
0x000000000041e0e1 <+177>: kxnorw %k0,%k0,%k1
0x000000000041e0e5 <+181>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e0f0 <+192>: vmovdqa -0x202d58(%rip),%xmm1 #
0x21b3a0
0x000000000041e0f8 <+200>: kxnorw %k0,%k0,%k1
0x000000000041e0fc <+204>: vpscatterdq %ymm0,0x55f180(,%xmm1,1){%k1}
0x000000000041e107 <+215>: movq $0x0,0x1414ee(%rip) #
0x55f600 <y_regs+1152>
0x000000000041e112 <+226>: pop %rbp
0x000000000041e113 <+227>: vzeroupper
0x000000000041e116 <+230>: ret
End of assembler dump.
Example is from 'cc -c -I. -Iproto -DHAVE_CONFIG_H -DLIBICONV_PLUG
-I/usr/local/include -O2 -pipe -march=skylake-avx512 -DLIBICONV_PLUG
-fstack-protector-strong -fno-strict-aliasing -D_REENTRANT
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -o objects/alloc.o alloc.c'
with CPU:
Intel(R) Xeon(R) CPU E3-1275 v6 @ 3.80GHz (3800.00-MHz K8-class CPU)
When I 1st noticed, I checked also much more recent CPUs - with the same
result - if CPUTUPE is set to any avx-involving optimization together
with -O2, binaries crash with SIGILL on Haswell, SkyLake, TigerLake...
My last world/ports builds (sucessfully with CPUTYPE?=ivybridge) were
with stable-13 (arround 13.1) - and llvm 13 I think - can't remember
exactly.
Thanks,
-harry
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?62cc8fed-6acd-45fb-a138-ac7cd218191a>