Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 9 Jul 2021 22:41:00 -0400
From:      Karl Denninger <karl@denninger.net>
To:        stable@freebsd.org
Subject:   Re: 12.2 Splay Tree ipfw potential panic source
Message-ID:  <7bfda38b-cf81-d8be-7691-e18946e6b56e@denninger.net>
In-Reply-To: <ca5beb7c-db38-1d3c-0f3c-b1b6a12c311e@denninger.net>
References:  <2e3dcd4d-c8e6-8381-0010-d0844c99901e@denninger.net> <20210708221134.GA32658@belenus.iks-jena.de> <a6a9c220-fee6-a0ea-7721-f88ff865a6a8@denninger.net> <CAFMmRNy9K-1mTDoqQhgdChWV5f_n4QhNesz%2B6xWywn_TQ43xng@mail.gmail.com> <ca5beb7c-db38-1d3c-0f3c-b1b6a12c311e@denninger.net>

next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.

--------------ms060608010600000502000206
Content-Type: multipart/alternative;
 boundary="------------4FA32C454518EECB922BCACF"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------4FA32C454518EECB922BCACF
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 7/9/2021 18:06, Karl Denninger wrote:
> On 7/9/2021 16:17, Ryan Stone wrote:
>> On Thu, Jul 8, 2021 at 8:54 PM Karl Denninger <karl@denninger.net>=20
>> wrote:
>>> I will see if I can get at least a panic backtrace, although the
>>> impacted box is a pcEngines firewall that boots of an SD card.
>> Have you checked whether netdump supports your NICs?=C2=A0 You should =
be
>> able to get a full vmcore off if so.
>
> Yes; the box in question is in heavy production and I will not be able =

> to get an isolated period of time to pull a core (assuming the remote=20
> dump works) until sometime this weekend.
>
> Will advise once I (hopefully) have it.
>
Ok, so I have good news and bad news.

I have the trap and it is definitely in libalias which appears to come=20
about as a result of a NAT translation attempt.

Fatal trap 18: integer divide fault while in kernel mode
cpuid =3D 1; apic id =3D 01
instruction pointer=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0x20:0xffffffff8275b7cc
stack pointer=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 =3D 0x28:0xfffffe0017b6b310
frame pointer=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 =3D 0x28:0xfffffe0017b6b320
code segment=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 =3D base 0x0, limit 0xfffff, type 0x1b
 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D DP=
L 0, pres 1, long 1, def32 0, gran 1
processor eflags=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D interrupt =
enabled, resume, IOPL =3D 0
current process=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0 (if=
_io_tqg_1)
trap number=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 =3D 18
panic: integer divide fault
cpuid =3D 1
time =3D 1625883072
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame=20
0xfffffe0017b6b020
vpanic() at vpanic+0x17b/frame 0xfffffe0017b6b070
panic() at panic+0x43/frame 0xfffffe0017b6b0d0
trap_fatal() at trap_fatal+0x391/frame 0xfffffe0017b6b130
trap() at trap+0x67/frame 0xfffffe0017b6b240
calltrap() at calltrap+0x8/frame 0xfffffe0017b6b240
--- trap 0x12, rip =3D 0xffffffff8275b7cc, rsp =3D 0xfffffe0017b6b310, rb=
p =3D=20
0xfffffe0017b6b320 ---
HouseKeeping() at HouseKeeping+0x1c/frame 0xfffffe0017b6b320
LibAliasInLocked() at LibAliasInLocked+0x2f/frame 0xfffffe0017b6b3e0
LibAliasIn() at LibAliasIn+0x46/frame 0xfffffe0017b6b410
ipfw_nat() at ipfw_nat+0x234/frame 0xfffffe0017b6b460
ipfw_chk() at ipfw_chk+0x1350/frame 0xfffffe0017b6b670
ipfw_check_packet() at ipfw_check_packet+0xf0/frame 0xfffffe0017b6b760
pfil_run_hooks() at pfil_run_hooks+0xb0/frame 0xfffffe0017b6b7f0
ip_input() at ip_input+0x427/frame 0xfffffe0017b6b8a0
netisr_dispatch_src() at netisr_dispatch_src+0xca/frame 0xfffffe0017b6b8f=
0
ether_demux() at ether_demux+0x138/frame 0xfffffe0017b6b920
ether_nh_input() at ether_nh_input+0x33b/frame 0xfffffe0017b6b980
netisr_dispatch_src() at netisr_dispatch_src+0xca/frame 0xfffffe0017b6b9d=
0
ether_input() at ether_input+0x4b/frame 0xfffffe0017b6ba00
iflib_rxeof() at iflib_rxeof+0xad6/frame 0xfffffe0017b6bae0
_task_fn_rx() at _task_fn_rx+0x72/frame 0xfffffe0017b6bb20
gtaskqueue_run_locked() at gtaskqueue_run_locked+0x121/frame=20
0xfffffe0017b6bb80
gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xb6/frame=20
0xfffffe0017b6bbb0
fork_exit() at fork_exit+0x7e/frame 0xfffffe0017b6bbf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0017b6bbf0
--- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 ---
Uptime: 7m23s
netdump: overwriting mbuf zone pointers
netdump in progress. searching for server...
netdumping to 192.168.10.100 (ac:1f:6b:ad:d8:cb)
Dumping 190 out of 1882 MB:. . . . . . . . . . . . .
** DUMP FAILED (ERROR 60) **

Now the bad news -- as you can see, an attempted remote dump fails,=20
possibly because the network code at that point is hosed. I get a 69632=20
length file (exactly and repeatedly) on the remote machine where the=20
dump is set to go; it looks like the first piece of it is indeed=20
received but that's it and then the panic'd unit reboots.

On the server (remote) end I have this in the "info" file:

Dump from IpGw [192.168.10.200]
Dump incomplete: client timed out

So it looks like it got the first part of it, the server replied but the =

crashed box never sent anything else.

-rw-------=C2=A0=C2=A0 1 root=C2=A0 wheel=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2=
 Jul=C2=A0 9 22:11 bounds.IpGw
-rw-------=C2=A0=C2=A0 1 root=C2=A0 wheel=C2=A0=C2=A0=C2=A0=C2=A0 66 Jul=C2=
=A0 9 22:10 info.IpGw.0
-rw-------=C2=A0=C2=A0 1 root=C2=A0 wheel=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=
 Jul=C2=A0 9 22:11 info.IpGw.1
-rw-------=C2=A0=C2=A0 1 root=C2=A0 wheel=C2=A0 69632 Jul=C2=A0 9 22:00 v=
mcore.IpGw.0
-rw-------=C2=A0=C2=A0 1 root=C2=A0 wheel=C2=A0 69632 Jul=C2=A0 9 22:11 v=
mcore.IpGw.1

Without a complete core I can't give you a good traceback.=C2=A0 I may be=
=20
able to get a local device on this unit sometime over the weekend=20
sometime -- not sure as of yet as it is in production use.

This is an extremely reliable panic -- uptime is only a few minutes=20
before it blows up.

--=20
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/

--------------4FA32C454518EECB922BCACF--

--------------ms060608010600000502000206
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC
DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL
MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw
FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf
BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4
MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD
dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1
ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI
KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD
0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY
vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn
uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24
SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E
6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH
YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL
h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd
zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE
FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q
EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ
TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5
c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY
MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC
AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN
gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9
oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj
tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K
uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv
HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK
17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/
Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA
6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY
UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO
62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp
ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD
QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx
MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw
FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A
16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg
96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg
y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs
YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg
6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX
SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM
cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN
5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l
CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg
CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v
b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF
oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG
SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O
BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC
GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE
BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh
IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ
zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN
AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR
dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu
2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X
lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK
FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q
IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA
1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL
shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ
qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE
QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL
MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM
TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM
QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw
GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjEwNzEwMDI0MTAw
WjBPBgkqhkiG9w0BCQQxQgRAP3Klgmpa1kctKVqfx0bF5eclItSpkCmzQxzVTlpg2PJWlPEJ
eZH3YzQPSBuIVYBZA+rn5Qya0RSg+sfRT5RMzzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl
AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3
DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL
MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM
TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM
QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg
gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz
dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0
ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF
AASCAgBxBKyMqAhZqnrr836e9z28E7bm2V3Im3h/ZneaxFDmqk/MvU6MvVcjOuRgKmo8SUTn
IvPUGdP5FkgpVxJkXN8lvabooiEh95C6y6BcDgcI7r9QhCP9OCdi++7l4vBcE3qWYKkI7EEK
dM8i4LKgMo4UYDKEZO+sh6Redx9AyUPtNJmReKOPE1CylgtgF7hAKnwFQhO7UG2PZdNzOvrZ
big5THGKU5aBTZ2zFdH8HHmWG7DT/I+oqXndtx5chIZg2M3wvWyQoL9jHsGOR1OMUr0QB4w5
/iZMWbn2ve48CMmUbDRM8BuK0w5M3Um1vvcraezSqaohP6WZs7xa0YUdv4HkJuy0ukoBRJZn
wF1F/ZY5sejR653p4+CkYZE4QjVBAxHqQkysHhFM9TjRLsJ8rts4WuwX553op/loW/cVdGAT
gs0P7H1IihV3apP2LvIxMNwcSXhRXQNuQHfZ+7hZgI/iCfJqNQs0ufn9U94NSJCFJ6Of4wlk
CPtd6aG2aV2oLvpya5rSuu8f38NY/2MH8hqlUpForpSlj37UqCG6mlunuF9az9gGZnmI6cVe
ef3h61Ow5uoguNQ31fap1mChGMxKvCwUHDDc8/9ztUK+Q88ZJZlrd2xo0h+QPi2qe22giW7J
jjLzwkoDP5Z81FjftQAOvF7IwTFeOfuc92BMff1OMwAAAAAAAA==
--------------ms060608010600000502000206--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7bfda38b-cf81-d8be-7691-e18946e6b56e>