Date: Wed, 7 May 2025 10:29:54 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: Mike Belanger <mibelanger@qnx.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Gleb Smirnoff <glebius@FreeBSD.org> Subject: Re: [EXTERNAL] - Re: Race condition in ether_ifattach Message-ID: <BD551667-A9CB-4E69-9868-FE680FDC0653@FreeBSD.org> In-Reply-To: <YQXPR01MB419836CF7ECD4C313912810CC28E2@YQXPR01MB4198.CANPRD01.PROD.OUTLOOK.COM> References: <YQXPR01MB41989535A01FA09637C82906C2822@YQXPR01MB4198.CANPRD01.PROD.OUTLOOK.COM> <7FFF346E-3205-49A9-B95A-94A418A28220@FreeBSD.org> <YQXPR01MB419836CF7ECD4C313912810CC28E2@YQXPR01MB4198.CANPRD01.PROD.OUTLOOK.COM>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On May 5, 2025, at 9:54 PM, Mike Belanger <mibelanger@qnx.com> wrote: > > In our reported case a startup script is loading the driver and bringing the interface up with ifconfig. > Since they are putting these commands to the background, so ifconfig is not properly waiting for the driver load to fully complete. > When ifconfig is successful, it will send the IPv6 neighbour discovery packets…and this can result in a crash if ether_ifattach is not complete (ifp->if_output is NULL). I think I see the problem. > We are considering breaking up if_attach_internal, so that ether_ifattach can call the first part and then call the end part after the ifp is fully setup. > We can reproduce the issue by adding an artificial delay after the if_attach in ether_ifattach. > > Mike. > > > From: owner-freebsd-net@FreeBSD.org <mailto:owner-freebsd-net@FreeBSD.org> <owner-freebsd-net@FreeBSD.org <mailto:owner-freebsd-net@FreeBSD.org>> on behalf of Zhenlei Huang <zlei@FreeBSD.org <mailto:zlei@FreeBSD.org>> > Date: Saturday, May 3, 2025 at 9:34 PM > To: Mike Belanger <mibelanger@qnx.com <mailto:mibelanger@qnx.com>> > Cc: freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> <freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org>>, Gleb Smirnoff <glebius@FreeBSD.org <mailto:glebius@FreeBSD.org>> > Subject: [EXTERNAL] - Re: Race condition in ether_ifattach > > CAUTION - This email is from an external source. Please be cautious with links and attachments. (go/taginfo) > > Hi Mike, > > > On May 1, 2025, at 9:13 PM, Mike Belanger <mibelanger@qnx.com <mailto:mibelanger@qnx.com>> wrote: > > There appears to be a race condition in ether_ifattach (if_ethersubr.c). > The ether_ifattach() function calls if_attach, where the interface will get announced, and then ether_ifattach continues with the initialization of the ifp. > > I also noticed this while working on https://reviews.freebsd.org/D49359 <https://urldefense.com/v3/__https:/reviews.freebsd.org/D49359__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ3FMtaxAA$>. There's an attempt for the attaching process https://reviews.freebsd.org/D49358 <https://urldefense.com/v3/__https:/reviews.freebsd.org/D49358__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ30mbVejw$>; . > > > then ether_ifattach continues with the initialization of the ifp. > In most cases that should not matter, as at that moment the interface has not been flagged up ( IFF_UP ) yet. > > > Is there any guarantee in FreeBSD that this race condition cannot be exposed. > We have been running the FreeBSD stack for some time under QNX and have just recently run into an issue with this race condition. > We are considering a modification where we have the option of deferring the interface announcement in if_attach. > > Can you elaborate how the race condition happens and how that affect you ? > > Before opening a FreeBSD bug, I wanted to check if this issue would not be valid in a FreeBSD system. > It’s very clear that there is a potential race when looking at the code, but perhaps there is a mitigation that is not obvious. > This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. > > Best regards, > Zhenlei > > This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation. Best regards, Zhenlei [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On May 5, 2025, at 9:54 PM, Mike Belanger <<a href="mailto:mibelanger@qnx.com" class="">mibelanger@qnx.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">In our reported case a startup script is loading the driver and bringing the interface up with ifconfig.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">Since they are putting these commands to the background, so ifconfig is not properly waiting for the driver load to fully complete.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">When ifconfig is successful, it will send the IPv6 neighbour discovery packets…and this can result in a crash if ether_ifattach is not complete (ifp->if_output is NULL).</span></div></div></div></blockquote><div><br class=""></div><div>I think I see the problem.</div><br class=""><blockquote type="cite" class=""><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">We are considering breaking up if_attach_internal, so that ether_ifattach can call the first part and then call the end part after the ifp is fully setup.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">We can reproduce the issue by adding an artificial delay after the if_attach in ether_ifattach.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">Mike.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div id="mail-editor-reference-message-container" class=""><div class=""><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm;" class=""><p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 10pt; font-family: Aptos, sans-serif;"><b class=""><span style="font-size: 12pt;" class="">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;" class=""><a href="mailto:owner-freebsd-net@FreeBSD.org" style="color: blue; text-decoration: underline;" class="">owner-freebsd-net@FreeBSD.org</a><span class="Apple-converted-space"> </span><<a href="mailto:owner-freebsd-net@FreeBSD.org" style="color: blue; text-decoration: underline;" class="">owner-freebsd-net@FreeBSD.org</a>> on behalf of Zhenlei Huang <<a href="mailto:zlei@FreeBSD.org" style="color: blue; text-decoration: underline;" class="">zlei@FreeBSD.org</a>><br class=""><b class="">Date:<span class="Apple-converted-space"> </span></b>Saturday, May 3, 2025 at 9:34</span><span style="font-size: 12pt; font-family: Arial, sans-serif;" class=""> </span><span style="font-size: 12pt;" class="">PM<br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>Mike Belanger <<a href="mailto:mibelanger@qnx.com" style="color: blue; text-decoration: underline;" class="">mibelanger@qnx.com</a>><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b><a href="mailto:freebsd-net@freebsd.org" style="color: blue; text-decoration: underline;" class="">freebsd-net@freebsd.org</a><span class="Apple-converted-space"> </span><<a href="mailto:freebsd-net@freebsd.org" style="color: blue; text-decoration: underline;" class="">freebsd-net@freebsd.org</a>>, Gleb Smirnoff <<a href="mailto:glebius@FreeBSD.org" style="color: blue; text-decoration: underline;" class="">glebius@FreeBSD.org</a>><br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>[EXTERNAL] - Re: Race condition in ether_ifattach<o:p class=""></o:p></span></p></div><div class=""><div align="center" class=""><table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" style="background-color: rgb(198, 170, 50);"><tbody class=""><tr class=""><td width="100%" style="width: 571px; padding: 0cm;" class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif; text-align: center;" class=""><b class=""><span style="font-size: 12pt;" class="">CAUTION</span></b><span style="font-size: 12pt;" class=""><span class="Apple-converted-space"> </span>- This email is from an external source. Please be cautious with links and attachments. (go/taginfo)</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></td></tr></tbody></table></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div></div><div class=""><div class=""><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">Hi Mike,<o:p class=""></o:p></span></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><br class=""><br class=""><o:p class=""></o:p></span></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">On May 1, 2025, at 9:13 PM, Mike Belanger <</span><a href="mailto:mibelanger@qnx.com" style="color: blue; text-decoration: underline;" class=""><span style="font-size: 12pt;" class="">mibelanger@qnx.com</span></a><span style="font-size: 12pt;" class="">> wrote:<o:p class=""></o:p></span></div></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div><div class=""><div class=""><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">There appears to be a race condition in ether_ifattach (if_ethersubr.c).</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">The ether_ifattach() function calls if_attach, where the interface will get announced, and then ether_ifattach continues with the initialization of the ifp.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div></div></div></blockquote><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">I also noticed this while working on </span><a href="https://urldefense.com/v3/__https:/reviews.freebsd.org/D49359__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ3FMtaxAA$" style="color: blue; text-decoration: underline;" class=""><span style="font-size: 12pt;" class="">https://reviews.freebsd.org/D49359</span></a><span style="font-size: 12pt;" class="">. There's an attempt for the attaching process </span><a href="https://urldefense.com/v3/__https:/reviews.freebsd.org/D49358__;!!JoeW-IhCUkS0Jg!Z0amzfdzApROIkoPw2gfHT4AlRbNoJhjhYrxU6fH_KH9W8eXaWsowj9sKZ0EvnqPG0to66NlKZ30mbVejw$" style="color: blue; text-decoration: underline;" class=""><span style="font-size: 12pt;" class="">https://reviews.freebsd.org/D49358</span></a><span style="font-size: 12pt;" class=""><span class="Apple-converted-space"> </span>.<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">> </span><span style="font-size: 11pt;" class="">then ether_ifattach continues with the initialization of the ifp.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">In most cases that should not matter, as at that moment the interface has not been flagged up ( IFF_UP ) yet.<o:p class=""></o:p></span></div></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><br class=""><br class=""><o:p class=""></o:p></span></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div class=""><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">Is there any guarantee in FreeBSD that this race condition cannot be exposed.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">We have been running the FreeBSD stack for some time under QNX and have just recently run into an issue with this race condition.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">We are considering a modification where we have the option of deferring the interface announcement in if_attach.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div></div></div></blockquote><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">Can you elaborate how the race condition happens and how that affect you ?<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div class=""><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">Before opening a FreeBSD bug, I wanted to check if this issue would not be valid in a FreeBSD system.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 11pt;" class="">It’s very clear that there is a potential race when looking at the code, but perhaps there is a mitigation that is not obvious.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div></div><div class="MsoNormal" align="center" style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif; text-align: center;"><span style="font-size: 12pt;" class=""><hr size="0" width="100%" align="center" class=""></span></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-family: Helvetica;" class="">This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.</span><span style="font-size: 12pt;" class=""><o:p class=""></o:p></span></div></div></blockquote></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div><div class=""><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">Best regards,<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class="">Zhenlei<o:p class=""></o:p></span></div></div></div><div style="margin: 0cm; font-size: 10pt; font-family: Aptos, sans-serif;" class=""><span style="font-size: 12pt;" class=""><o:p class=""> </o:p></span></div></div></div></div></div></div></div></div><hr style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">This email and any attachments are intended solely for the use of the individual or entity to whom they are addressed. This email may contain information that is confidential, privileged, or otherwise protected from disclosure. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this email in error, please immediately contact the sender and delete all copies of this email and any attachments from your systems. Any unauthorized review, use, dissemination, distribution, or reproduction of this email by unintended recipients is not authorized and may be unlawful. Thank you for your cooperation.</span></div></blockquote></div><br class=""><div class=""> <div>Best regards,</div><div>Zhenlei</div> </div> <br class=""></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BD551667-A9CB-4E69-9868-FE680FDC0653>