Date: Fri, 9 May 2025 13:15:00 +0000 From: Mike Belanger <mibelanger@qnx.com> To: Zhenlei Huang <zlei@FreeBSD.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Gleb Smirnoff <glebius@FreeBSD.org> Subject: Re: [EXTERNAL] - Re: Race condition in ether_ifattach Message-ID: <YQXPR01MB4198790AC4C71557DC4B03B0C28AA@YQXPR01MB4198.CANPRD01.PROD.OUTLOOK.COM> In-Reply-To: <BD551667-A9CB-4E69-9868-FE680FDC0653@FreeBSD.org> References: <YQXPR01MB41989535A01FA09637C82906C2822@YQXPR01MB4198.CANPRD01.PROD.OUTLOOK.COM> <7FFF346E-3205-49A9-B95A-94A418A28220@FreeBSD.org> <YQXPR01MB419836CF7ECD4C313912810CC28E2@YQXPR01MB4198.CANPRD01.PROD.OUTLOOK.COM> <BD551667-A9CB-4E69-9868-FE680FDC0653@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Let me know if there is anything else I can do for this issue (e.g. open a bug). From: Zhenlei Huang <zlei@FreeBSD.org> Date: Tuesday, May 6, 2025 at 10:30 PM To: Mike Belanger <mibelanger@qnx.com> Cc: freebsd-net@freebsd.org <freebsd-net@freebsd.org>, Gleb Smirnoff <glebius@FreeBSD.org> Subject: Re: [EXTERNAL] - Re: Race condition in ether_ifattach CAUTION - This email is from an external source. Please be cautious with links and attachments. (go/taginfo) On May 5, 2025, at 9:54 PM, Mike Belanger <mibelanger@qnx.com<mailto:mibelanger@qnx.com>> wrote: In our reported case a startup script is loading the driver and bringing the interface up with ifconfig. Since they are putting these commands to the background, so ifconfig is not properly waiting for the driver load to fully complete. When ifconfig is successful, it will send the IPv6 neighbour discovery packets…and this can result in a crash if ether_ifattach is not complete (ifp->if_output is NULL). I think I see the problem. We are considering breaking up if_attach_internal, so that ether_ifattach can call the first part and then call the end part after the ifp is fully setup. We can reproduce the issue by adding an artificial delay after the if_attach in ether_ifattach. Mike. From: owner-freebsd-net@FreeBSD.org<mailto:owner-freebsd-net@FreeBSD.org> <owner-freebsd-net@FreeBSD.org<mailto:owner-freebsd-net@FreeBSD.org>> on behalf of Zhenlei Huang <zlei@FreeBSD.org<mailto:zlei@FreeBSD.org>> Date: Saturday, May 3, 2025 at 9:34 PM To: Mike Belanger <mibelanger@qnx.com<mailto:mibelanger@qnx.com>> Cc: freebsd-net@freebsd.org<mailto:freebsd-net@freebsd.org> <freebsd-net@freebsd.org<mailto:freebsd-net@freebsd.org>>, Gleb Smirnoff <glebius@FreeBSD.org<mailto:glebius@FreeBSD.org>> Subject: [EXTERNAL] - Re: Race condition in ether_ifattach CAUTION - This email is from an external source. Please be cautious with links and attachments. (go/taginfo) Hi Mike, On May 1, 2025, at 9:13 PM, Mike Belanger <mibelanger@qnx.com<mailto:mibelanger@qnx.com>> wrote: There appears to be a race condition in ether_ifattach (if_ethersubr.c). The ether_ifattach() function calls if_attach, where the interface will get announced, and then ether_ifattach continues with the initialization of the ifp. I also noticed this while working on https://reviews.freebsd.org/D49359<https://urldefense.com/v3/__https:/reviews.freebsd.org/D49359__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ3FMtaxAA$>. There's an attempt for the attaching process https://reviews.freebsd.org/D49358<https://urldefense.com/v3/__https:/reviews.freebsd.org/D49358__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ30mbVejw$>; . > then ether_ifattach continues with the initialization of the ifp. In most cases that should not matter, as at that moment the interface has not been flagged up ( IFF_UP ) yet. Is there any guarantee in FreeBSD that this race condition cannot be exposed. We have been running the FreeBSD stack for some time under QNX and have just recently run into an issue with this race condition. We are considering a modification where we have the option of deferring the interface announcement in if_attach. Can you elaborate how the race condition happens and how that affect you ? Before opening a FreeBSD bug, I wanted to check if this issue would not be valid in a FreeBSD system. It’s very clear that there is a potential race when looking at the code, but perhaps there is a mitigation that is not obvious. ________________________________ This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. Best regards, Zhenlei ________________________________ This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation. Best regards, Zhenlei ---------------------------------------------------------------------- This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation. [-- Attachment #2 --] <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">; <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <!--[if !mso]><style>v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} </style><![endif]--><style><!-- /* Font Definitions */ @font-face {font-family:Helvetica; panose-1:0 0 0 0 0 0 0 0 0 0;} @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Aptos; panose-1:2 11 0 4 2 2 2 2 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:10.0pt; font-family:"Aptos",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} span.apple-converted-space {mso-style-name:apple-converted-space;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt; mso-ligatures:none;} @page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext="edit" spidmax="1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext="edit"> <o:idmap v:ext="edit" data="1" /> </o:shapelayout></xml><![endif]--> </head> <body lang="EN-CA" link="blue" vlink="purple" style="word-wrap:break-word;line-break:after-white-space"> <div class="WordSection1"> <p class="MsoNormal"><span style="font-size:11.0pt">Let me know if there is anything else I can do for this issue (e.g. open a bug).<o:p></o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p> <div id="mail-editor-reference-message-container"> <div> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Zhenlei Huang <zlei@FreeBSD.org><br> <b>Date: </b>Tuesday, May 6, 2025 at 10:30</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:12.0pt;color:black">PM<br> <b>To: </b>Mike Belanger <mibelanger@qnx.com><br> <b>Cc: </b>freebsd-net@freebsd.org <freebsd-net@freebsd.org>, Gleb Smirnoff <glebius@FreeBSD.org><br> <b>Subject: </b>Re: [EXTERNAL] - Re: Race condition in ether_ifattach<o:p></o:p></span></p> </div> <div> <div align="center"> <table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" style="background:#C6AA32"> <tbody> <tr> <td width="100%" style="width:100.0%;padding:0cm 0cm 0cm 0cm"> <p class="MsoNormal" align="center" style="text-align:center"><b><span style="font-size:12.0pt;color:black">CAUTION</span></b><span style="font-size:12.0pt;color:black"> - This email is from an external source. Please be cautious with links and attachments. (go/taginfo)</span><span style="font-size:12.0pt"><o:p></o:p></span></p> </td> </tr> </tbody> </table> </div> <p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p> </div> <p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p> <div> <p class="MsoNormal"><span style="font-size:12.0pt"><br> <br> <o:p></o:p></span></p> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <div> <p class="MsoNormal"><span style="font-size:12.0pt">On May 5, 2025, at 9:54 PM, Mike Belanger <</span><a href="mailto:mibelanger@qnx.com"><span style="font-size:12.0pt">mibelanger@qnx.com</span></a><span style="font-size:12.0pt">> wrote:<o:p></o:p></span></p> </div> <p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p> <div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">In our reported case a startup script is loading the driver and bringing the interface up with ifconfig.</span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">Since they are putting these commands to the background, so ifconfig is not properly waiting for the driver load to fully complete.</span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">When ifconfig is successful, it will send the IPv6 neighbour discovery packets…and this can result in a crash if ether_ifattach is not complete (ifp->if_output is NULL).</span></p> </div> </div> </div> </blockquote> <div> <p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p> </div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">I think I see the problem.<o:p></o:p></span></p> </div> <p class="MsoNormal"><span style="font-size:12.0pt"><br> <br> <o:p></o:p></span></p> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">We are considering breaking up if_attach_internal, so that ether_ifattach can call the first part and then call the end part after the ifp is fully setup.</span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">We can reproduce the issue by adding an artificial delay after the if_attach in ether_ifattach.</span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt"> </span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">Mike.</span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt"> </span></p> </div> <div> <p class="MsoNormal"><span style="font-size:11.0pt"> </span></p> </div> <div id="mail-editor-reference-message-container"> <div> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt">From:<span class="apple-converted-space"> </span></span></b><a href="mailto:owner-freebsd-net@FreeBSD.org"><span style="font-size:12.0pt">owner-freebsd-net@FreeBSD.org</span></a><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt"><</span><a href="mailto:owner-freebsd-net@FreeBSD.org"><span style="font-size:12.0pt">owner-freebsd-net@FreeBSD.org</span></a><span style="font-size:12.0pt">> on behalf of Zhenlei Huang <</span><a href="mailto:zlei@FreeBSD.org"><span style="font-size:12.0pt">zlei@FreeBSD.org</span></a><span style="font-size:12.0pt">><br> <b>Date:<span class="apple-converted-space"> </span></b>Saturday, May 3, 2025 at 9:34</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:12.0pt">PM<br> <b>To:<span class="apple-converted-space"> </span></b>Mike Belanger <</span><a href="mailto:mibelanger@qnx.com"><span style="font-size:12.0pt">mibelanger@qnx.com</span></a><span style="font-size:12.0pt">><br> <b>Cc:<span class="apple-converted-space"> </span></b></span><a href="mailto:freebsd-net@freebsd.org"><span style="font-size:12.0pt">freebsd-net@freebsd.org</span></a><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt"><</span><a href="mailto:freebsd-net@freebsd.org"><span style="font-size:12.0pt">freebsd-net@freebsd.org</span></a><span style="font-size:12.0pt">>, Gleb Smirnoff <</span><a href="mailto:glebius@FreeBSD.org"><span style="font-size:12.0pt">glebius@FreeBSD.org</span></a><span style="font-size:12.0pt">><br> <b>Subject:<span class="apple-converted-space"> </span></b>[EXTERNAL] - Re: Race condition in ether_ifattach</span><o:p></o:p></p> </div> <div> <div align="center"> <table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" style="background:#C6AA32"> <tbody> <tr> <td width="100%" style="width:100.0%;padding:0cm 0cm 0cm 0cm"> <p class="MsoNormal" align="center" style="text-align:center"><b><span style="font-size:12.0pt;color:black">CAUTION</span></b><span class="apple-converted-space"><span style="font-size:12.0pt;color:black"> </span></span><span style="font-size:12.0pt;color:black">- This email is from an external source. Please be cautious with links and attachments. (go/taginfo)</span></p> </td> </tr> </tbody> </table> </div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> </div> <div> <div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">Hi Mike,</span></p> </div> <div> <div> <p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">On May 1, 2025, at 9:13 PM, Mike Belanger <</span><a href="mailto:mibelanger@qnx.com"><span style="font-size:12.0pt">mibelanger@qnx.com</span></a><span style="font-size:12.0pt">> wrote:</span></p> </div> </div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> <div> <div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">There appears to be a race condition in ether_ifattach (if_ethersubr.c).</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">The ether_ifattach() function calls if_attach, where the interface will get announced, and then ether_ifattach continues with the initialization of the ifp.</span></p> </div> </div> </div> </div> </blockquote> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">I also noticed this while working on </span><a href="https://urldefense.com/v3/__https:/reviews.freebsd.org/D49359__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ3FMtaxAA$"><span style="font-size:12.0pt">https://reviews.freebsd.org/D49359</span></a><span style="font-size:12.0pt">. There's an attempt for the attaching process </span><a href="https://urldefense.com/v3/__https:/reviews.freebsd.org/D49358__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ30mbVejw$"><span style="font-size:12.0pt">https://reviews.freebsd.org/D49358</span></a><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">.</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">> </span><span style="font-size:11.0pt">then ether_ifattach continues with the initialization of the ifp.</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">In most cases that should not matter, as at that moment the interface has not been flagged up ( IFF_UP ) yet.</span></p> </div> </div> <div> <p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <div> <div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">Is there any guarantee in FreeBSD that this race condition cannot be exposed.</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">We have been running the FreeBSD stack for some time under QNX and have just recently run into an issue with this race condition.</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">We are considering a modification where we have the option of deferring the interface announcement in if_attach.</span></p> </div> </div> </div> </div> </blockquote> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">Can you elaborate how the race condition happens and how that affect you ?</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <div> <div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">Before opening a FreeBSD bug, I wanted to check if this issue would not be valid in a FreeBSD system.</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:11.0pt">It’s very clear that there is a potential race when looking at the code, but perhaps there is a mitigation that is not obvious.</span></p> </div> </div> </div> <div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt"> <hr size="0" width="100%" align="center"> </span></div> <div> <p class="MsoNormal"><span style="font-family:Helvetica">This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.</span></p> </div> </div> </blockquote> </div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> <div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">Best regards,</span></p> </div> </div> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">Zhenlei</span></p> </div> </div> </div> <div> <p class="MsoNormal"><span style="font-size:12.0pt"> </span></p> </div> </div> </div> </div> </div> </div> </div> </div> <div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt"> <hr size="0" width="100%" align="center"> </span></div> <p class="MsoNormal"><span style="font-family:Helvetica">This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation.</span><span style="font-size:12.0pt"><o:p></o:p></span></p> </div> </blockquote> </div> <p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p> <div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">Best regards,<o:p></o:p></span></p> </div> <div> <p class="MsoNormal"><span style="font-size:12.0pt">Zhenlei<o:p></o:p></span></p> </div> </div> <p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p> </div> </div> </div> </div> <HR>This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation.<BR> </body> </html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YQXPR01MB4198790AC4C71557DC4B03B0C28AA>