Date: Wed, 24 Oct 2018 19:10:41 +0300 From: Lev Serebryakov <lev@FreeBSD.org> To: Ryan Stone <rysto32@gmail.com> Cc: freebsd-hackers@freebsd.org, Alan Somers <asomers@freebsd.org>, Conrad Meyer <cse.cem@gmail.com> Subject: Re: What is wrong with dtrace's stack()? Message-ID: <c7e9aa2f-8efa-6f48-d4b1-3705928803b2@FreeBSD.org> In-Reply-To: <CAFMmRNxbu=V6F9LW_6oWeQzws%2B1-UhETC%2BWWGMFGKAoH-5Ajjw@mail.gmail.com> References: <170994671.20181021201021@serebryakov.spb.ru> <CAG6CVpVCe-8OVU%2BKEyAh8swFSAjRshNcNLN0tjAS3W8bZxyrTA@mail.gmail.com> <475670271.20181022003734@serebryakov.spb.ru> <CAOtMX2goztCYJXM72WNR711DEBEKAGk1zFHF=NpE4UiVAF2bAQ@mail.gmail.com> <234d745d-37a9-9610-15b9-0f5cd5af21bf@FreeBSD.org> <b30f2080-1037-1752-60fb-6d22e28ce90d@FreeBSD.org> <CAFMmRNz35Ay82NcMnUkSAvpyU5p2m9ykiji1xXcbEanH9A2H%2Bw@mail.gmail.com> <168122586.20181024003412@serebryakov.spb.ru> <CAFMmRNxbu=V6F9LW_6oWeQzws%2B1-UhETC%2BWWGMFGKAoH-5Ajjw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--qAxpfnw6JzFvUEbu8htvvBpEoe9xyZ1h6
Content-Type: multipart/mixed; boundary="12uJTI8sAMiThSkLuym1l9QItCGS2mjQC";
protected-headers="v1"
From: Lev Serebryakov <lev@FreeBSD.org>
Reply-To: lev@FreeBSD.org
To: Ryan Stone <rysto32@gmail.com>
Cc: freebsd-hackers@freebsd.org, Alan Somers <asomers@freebsd.org>,
Conrad Meyer <cse.cem@gmail.com>
Message-ID: <c7e9aa2f-8efa-6f48-d4b1-3705928803b2@FreeBSD.org>
Subject: Re: What is wrong with dtrace's stack()?
References: <170994671.20181021201021@serebryakov.spb.ru>
<CAG6CVpVCe-8OVU+KEyAh8swFSAjRshNcNLN0tjAS3W8bZxyrTA@mail.gmail.com>
<475670271.20181022003734@serebryakov.spb.ru>
<CAOtMX2goztCYJXM72WNR711DEBEKAGk1zFHF=NpE4UiVAF2bAQ@mail.gmail.com>
<234d745d-37a9-9610-15b9-0f5cd5af21bf@FreeBSD.org>
<b30f2080-1037-1752-60fb-6d22e28ce90d@FreeBSD.org>
<CAFMmRNz35Ay82NcMnUkSAvpyU5p2m9ykiji1xXcbEanH9A2H+w@mail.gmail.com>
<168122586.20181024003412@serebryakov.spb.ru>
<CAFMmRNxbu=V6F9LW_6oWeQzws+1-UhETC+WWGMFGKAoH-5Ajjw@mail.gmail.com>
In-Reply-To: <CAFMmRNxbu=V6F9LW_6oWeQzws+1-UhETC+WWGMFGKAoH-5Ajjw@mail.gmail.com>
--12uJTI8sAMiThSkLuym1l9QItCGS2mjQC
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
On 24.10.2018 18:48, Ryan Stone wrote:
> sosend_generic+0x112c is the return address, so it's one instruction
> after the call instruction that called lock_delay.
Yes, but this instruction (+0x112b) calls real protocol send function
via function pointer.
> What's the line number of sosend_generic+0x112b?
It is here, uipc_socket.c:1589:
error =3D (*so->so_proto->pr_usrreqs->pru_send)(so,
(flags & MSG_OOB) ? PRUS_OOB :
/*
* If the user set MSG_EOF, the protocol understands
* this flag and nothing left to send then use
* PRU_SEND_EOF instead of PRU_SEND.
*/
((flags & MSG_EOF) &&
(so->so_proto->pr_flags & PR_IMPLOPCL) &&
(resid <=3D 0)) ?
PRUS_EOF :
/* If there is more to send set PRUS_MORETOCOME. */
(flags & MSG_MORETOCOME) ||
(resid > 0 && space > 0) ? PRUS_MORETOCOME : 0,
top, addr, control, td);
I have these three DIFFERENT stack, which should be one for sure, as it
is confirmed by custom SDT probes now (I've added them and checked).
Addresses are different from previous examples due to some code was
shifted by SDT probes. 26462 times stack is Ok, but 4068+3993 times some
frames are lost. Same address could not call both tcp_usr_send() and
ia32_pause() and lock_delay(). Really, it should be one stack trace!
kernel`lock_delay+0x72
kernel`sosend_generic+0xf61
kernel`sosend+0x79
kernel`soo_write+0x6b
kernel`fo_write+0x4a
kernel`dofilewrite+0xcd
kernel`kern_writev+0x79
kernel`sys_write+0x8f
kernel`syscallenter+0x774
kernel`amd64_syscall+0x1b
kernel`0xffffffff80cedfbd
3993
kernel`ia32_pause+0x7
kernel`sosend_generic+0xf61
kernel`sosend+0x79
kernel`soo_write+0x6b
kernel`fo_write+0x4a
kernel`dofilewrite+0xcd
kernel`kern_writev+0x79
kernel`sys_write+0x8f
kernel`syscallenter+0x774
kernel`amd64_syscall+0x1b
kernel`0xffffffff80cedfbd
4068
kernel`ia32_pause+0x6
kernel`tcp_usr_send+0x131
kernel`sosend_generic+0xf61
kernel`sosend+0x79
kernel`soo_write+0x6b
kernel`fo_write+0x4a
kernel`dofilewrite+0xcd
kernel`kern_writev+0x79
kernel`sys_write+0x8f
kernel`syscallenter+0x774
kernel`amd64_syscall+0x1b
kernel`0xffffffff80cedfbd
26462
> On Tue, Oct 23, 2018 at 5:34 PM Lev Serebryakov <lev@freebsd.org> wrote=
:
>>
>> Hello Ryan,
>>
>> Monday, October 22, 2018, 11:50:29 PM, you wrote:
>>
>>> Adding -fno-optimize-sibling-calls to the compiler flags will elimina=
te the TCO.
>> Stacks are better but still strange. For example I have such stack (w=
hich I
>> like better than previous):
>>
>> kernel`lock_delay+0x72
>> kernel`sosend_generic+0x112c
>> kernel`sosend+0x79
>> kernel`soo_write+0x6b
>> kernel`fo_write+0x4a
>> kernel`dofilewrite+0xcd
>> kernel`kern_writev+0x79
>> kernel`sys_write+0x8f
>> kernel`syscallenter+0x774
>> kernel`amd64_syscall+0x1b
>> kernel`0xffffffff80cebf6d
>>
>> According to addr2line `sosend_generic+0x112c' is
>>
>> https://svnweb.freebsd.org/base/head/sys/kern/uipc_socket.c?revision=3D=
339419&view=3Dmarkup#l1579
>>
>> Which is call to protocol-specific send function. Where is this funct=
ion
>> (it should be tcp for sure)?!
>>
>> --
>> Best regards,
>> Lev mailto:lev@FreeBSD.org
>>
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o=
rg"
>=20
--=20
// Lev Serebryakov
--12uJTI8sAMiThSkLuym1l9QItCGS2mjQC--
--qAxpfnw6JzFvUEbu8htvvBpEoe9xyZ1h6
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKTBAEBCgB9FiEE+W0coLX0MYtnSzMK6rA8WL/cR48FAlvQmYFfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEY5
NkQxQ0EwQjVGNDMxOEI2NzRCMzMwQUVBQjAzQzU4QkZEQzQ3OEYACgkQ6rA8WL/c
R48MaxAAyiUF7zIrHfmfyvijJJKVBzZk8dL+ko0sUuqonaHaP/BLk088KAabpKd+
4nqnolCcpRlmenenR54t6EvDKXAkhKvH4Qy52NAjU699uzqBDcnk4FKPpq2gKZbE
Dz7WUb8itARTeq7ztFoJ00VPsLZSFoAWipuhpwvDwM1U/J54Iprh7v3rYlW7LJNJ
88Cd/xsxjt9ZtNw/F6YbncsgAIrDxSSTdAUa2oztcHoyM5x7PqrsoXcjQ8Ln2Pir
ZNUFd8vI62z44IVHnuNRX/1mmWRBHFRkgnkfTKEimJIcK9ZfAccP4bS6OEJBYLLA
0c/xougcb7JJ58EWLXWQk1qzVxozGapWLozpWADSgc5AYi1g3xYlyA8Z9Qyy0emF
InmKWnGHe7JPYPd5VnyI8hABH3PbQKuelmfEKgytjDrhxKXL+EWupgw20MLJhZcG
2VKEvaenoYVGYUUhhPvZ10PMQ69ulmWelC2J4n1xOYNdFLa/M3/98re8RhySuci2
bd7uk/7UTTDFIl7/y1r06Nnwjqe6ozo6ooiIHd6U0y/54iXpHO+rEKkBikTrPFF6
tHSt+8rMoP0OKq/V2U7QTulLFDPnEaanFS1Z06tw1wAVnqL3vGHdoFpcxPY4MoYZ
a9w59fSX7iS4LAeUdlrwM5FMCGUx3gr5pYVJznzVEjOjOebrRTU=
=KiLK
-----END PGP SIGNATURE-----
--qAxpfnw6JzFvUEbu8htvvBpEoe9xyZ1h6--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c7e9aa2f-8efa-6f48-d4b1-3705928803b2>