Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 13 Mar 2024 21:17:32 +0800
From:      Zhenlei Huang <zlei@FreeBSD.org>
To:        Benoit Chesneau <benoitc@enki-multimedia.eu>
Cc:        "freebsd-net@FreeBSD.org" <freebsd-net@freebsd.org>, Alexander Chernikov <melifaro@freebsd.org>
Subject:   Re: ipv4 route with  ipv6 local link nexthop ?
Message-ID:  <BE4D70C1-885D-4C6B-AE8C-AC62BD450F8A@FreeBSD.org>
In-Reply-To: <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org>
References:  <Kj1GA9KKe3wVoeEJr0g-4BAiin5cEjGj4ICYDH9LHBT_vpmctoN-2HNS7AjTwNimOVkn5nBoJw50fbLgNdCleBD0p_TZImOet1gFGIfr4-E=@enki-multimedia.eu> <367504DC-48DA-4DFD-9DB6-CC571F0D26B8@FreeBSD.org> <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help

--Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8



> On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <zlei@FreeBSD.org> wrote:
>=20
>=20
>=20
>> On Mar 13, 2024, at 12:19 PM, Zhenlei Huang <zlei@FreeBSD.org =
<mailto:zlei@FreeBSD.org>> wrote:
>>=20
>>=20
>>=20
>>> On Mar 13, 2024, at 4:36 AM, Benoit Chesneau =
<benoitc@enki-multimedia.eu <mailto:benoitc@enki-multimedia.eu>> wrote:
>>>=20
>>> On latest freebsd 14.0 release , I am trying to join an ipv4 address =
over an IPV6 local link using the following command :
>>>=20
>>> `route add -net 10.200.1.1/32 -inet6 =
fe80::9439:36ff:fef0:7cbd%vlan200`
>>>=20
>>> But I  get the following error:
>>>=20
>>> ```
>>> route add -net 10.200.1.1/32 -inet6 =
fe80::9439:36ff:fef0:7cbd%vlan200
>>> add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 fib 0: =
Invalid argument
>>=20
>> I guess that is regression by transition of route(8) to NETLINK.
>>=20
>> Can you please build sbin/route from source without NETLINK and give =
it another try ?
>>=20
>> ```
>> # cd /usr/src/sbin/route
>> # setenv MK_NETLINK_SUPPORT no
>> # make
>> ```
>=20
> Hi Benoit,
>=20
> I managed to repeat this bug.
>=20
> As a workaround, add an IPv4 address to the outgoing interface =
(vlan200 in your case),
> ```
> # ifconfig vlan200 inet x.x.x.x/y
> ```
>=20
> then retry adding the route
> ```
> # route add -net 10.200.1.1/32 -inet6 =
fe80::9439:36ff:fef0:7cbd%vlan200
> ```
>=20

Hi Alexander,

I think the logic of adding route from NETLINK routine is not complete.

After bumping netlink debug level,
```
# sysctl net.netlink.debug.nl_route_debug_level=3D9
```

I got these from kernel message:
```
# dmesg | tail
...
[nl_route] finalize_nhop: Unable to determine ifa, skipping
[nl_route] rtnl_handle_newroute: Error creating nexthop
...
```

=46rom =
https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Dreleng/14.0#n=
753 :

```
	/* Both nh_ifp and gateway are set */
	if (nh->nh_ifa =3D=3D NULL) {
		const struct sockaddr *gw_sa =3D &nh->gw_sa;

		if (gw_sa->sa_family !=3D dst->sa_family) {
			/*
			 * Use dst as the target for determining the =
default
			 * preferred ifa IF
			 * 1) the gateway is link-level (e.g. direct =
route)
			 * 2) the gateway family is different (e.g. IPv4 =
over IPv6).
			 */
			gw_sa =3D dst;
		}

		struct ifaddr *ifa =3D ifaof_ifpforaddr(gw_sa, =
nh->nh_ifp);
		if (ifa =3D=3D NULL) {
			NL_LOG(LOG_DEBUG, "Unable to determine ifa, =
skipping");
			*perror =3D EINVAL;
			return (NULL);
		}
		nhop_set_src(nh, ifa);
	}
```
If the outgoing interface `nh->nh_ifp` does not have any IPv4 addresses, =
we should try ifaddrs on other interfaces,
and have 0.0.0.0 as the last sort ( in case the net stack is IPv6 only, =
i.e., no IPv4 addresses available )


>>=20
>>> ```
>>>=20
>>> ping6 to `fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing =
wrong? Configuration of the interface is the following:
>>>=20
>>> ```
>>> vlan200: =
flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric =
0 mtu 1500
>>>         =
options=3D4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,MEXT=
PG>
>>>         ether fe:d9:dd:2d:35:c5
>>>         inet6 fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid =
0x5
>>>         groups: vlan
>>>         vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent interface: =
iavf0
>>>         media: Ethernet autoselect (10Gbase-SR <full-duplex>)
>>>         status: active
>>>         nd6 options=3D23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL
>>> ```
>>>=20
>>>=20
>>>=20
>>> Beno=C3=AEt=20
>>=20
>> Best regards,
>> Zhenlei




--Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br =
class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div =
class=3D"">On Mar 13, 2024, at 9:09 PM, Zhenlei Huang &lt;<a =
href=3D"mailto:zlei@FreeBSD.org" class=3D"">zlei@FreeBSD.org</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><meta =
charset=3D"UTF-8" class=3D""><br class=3D"Apple-interchange-newline"><br =
class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; =
font-size: 13px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><blockquote type=3D"cite" class=3D"" style=3D"font-family: =
Helvetica; font-size: 13px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; =
-webkit-text-stroke-width: 0px; text-decoration: none;"><div class=3D"">On=
 Mar 13, 2024, at 12:19 PM, Zhenlei Huang &lt;<a =
href=3D"mailto:zlei@FreeBSD.org" class=3D"">zlei@FreeBSD.org</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div =
class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; =
font-size: 13px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><br class=3D"Apple-interchange-newline"><br class=3D""><blockquote =
type=3D"cite" class=3D""><div class=3D"">On Mar 13, 2024, at 4:36 AM, =
Benoit Chesneau &lt;<a href=3D"mailto:benoitc@enki-multimedia.eu" =
class=3D"">benoitc@enki-multimedia.eu</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;">On latest =
freebsd 14.0 release , I am trying to join an ipv4 address over an IPV6 =
local link using the following command :</div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><br =
class=3D""></div><div class=3D"" style=3D"font-family: Arial, =
sans-serif; font-size: 14px;"><span class=3D"">`route add -net =
10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200`</span><br =
class=3D""></div><div class=3D"" style=3D"font-family: Arial, =
sans-serif; font-size: 14px;"><span class=3D""><br =
class=3D""></span></div><div class=3D"" style=3D"font-family: Arial, =
sans-serif; font-size: 14px;"><span class=3D"">But I &nbsp;get the =
following error:</span></div><div class=3D"" style=3D"font-family: =
Arial, sans-serif; font-size: 14px;"><span class=3D""><br =
class=3D""></span></div><div class=3D"" style=3D"font-family: Arial, =
sans-serif; font-size: 14px;">```</div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span =
class=3D"">route add -net 10.200.1.1/32 -inet6 =
fe80::9439:36ff:fef0:7cbd%vlan200</span><br class=3D""><span =
class=3D"">add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 =
fib 0: Invalid argument</span><br class=3D""></div></div></blockquote><div=
 class=3D""><br class=3D""></div><div class=3D"">I guess that is =
regression by transition of route(8) to =
NETLINK.</div></div></div></blockquote><blockquote type=3D"cite" =
class=3D"" style=3D"font-family: Helvetica; font-size: 13px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><div class=3D""><div class=3D"" =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
13px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;"><div =
class=3D""><br class=3D""></div><div class=3D"">Can you please build =
sbin/route from source without NETLINK and give it another try =
?</div><div class=3D""><br class=3D""></div><div class=3D"">```</div><div =
class=3D""># cd /usr/src/sbin/route</div><div class=3D""># =
setenv&nbsp;MK_NETLINK_SUPPORT no</div><div class=3D""># make</div><div =
class=3D"">```</div></div></div></blockquote><div style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br class=3D""></div><div =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
13px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D"">Hi =
Benoit,</div><div style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 13px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><br class=3D""></div><div style=3D"caret-color: rgb(0, =
0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D"">I managed to repeat this =
bug.</div><div style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 13px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><br class=3D""></div><div style=3D"caret-color: rgb(0, =
0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><div class=3D"">As a workaround, add =
an IPv4 address to the outgoing interface (vlan200 in your =
case),</div><div class=3D"">```</div><div class=3D""># ifconfig vlan200 =
inet x.x.x.x/y</div><div class=3D"">```</div><div class=3D""><br =
class=3D""></div><div class=3D"">then retry adding the route</div><div =
class=3D"">```</div><div class=3D""># route add -net 10.200.1.1/32 =
-inet6 fe80::9439:36ff:fef0:7cbd%vlan200</div><div =
class=3D"">```</div><div class=3D""><br =
class=3D""></div></div></div></blockquote><div><br =
class=3D""></div><div>Hi Alexander,</div><div><br class=3D""></div><div>I =
think the logic of adding route from NETLINK routine is not =
complete.</div><div></div><div><br class=3D""></div><div><div>After =
bumping netlink debug level,</div><div>```</div><div># sysctl =
net.netlink.debug.nl_route_debug_level=3D9</div><div>```</div><div><br =
class=3D""></div><div>I got these from kernel =
message:</div><div>```</div><div># dmesg | =
tail</div><div>...</div><div>[nl_route] finalize_nhop: Unable to =
determine ifa, skipping</div><div>[nl_route] rtnl_handle_newroute: Error =
creating nexthop</div><div>...</div><div>```</div><div><br =
class=3D""></div><div>=46rom <a =
href=3D"https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Drelen=
g/14.0#n753" =
class=3D"">https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Dre=
leng/14.0#n753</a> :</div><div><br =
class=3D""></div><div>```</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">	</span>/* Both nh_ifp and gateway are =
set */</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">=
	</span>if (nh-&gt;nh_ifa =3D=3D NULL) {</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre">		=
</span>const struct sockaddr *gw_sa =3D &amp;nh-&gt;gw_sa;</div><div><br =
class=3D""></div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">		</span>if (gw_sa-&gt;sa_family =
!=3D dst-&gt;sa_family) {</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">			=
</span>/*</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">			</span> * Use dst as the =
target for determining the default</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre">			=
</span> * preferred ifa IF</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">			</span> * 1) the gateway =
is link-level (e.g. direct route)</div><div><span class=3D"Apple-tab-span"=
 style=3D"white-space:pre">			</span> * 2) the gateway =
family is different (e.g. IPv4 over IPv6).</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre">			=
</span> */</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">			</span>gw_sa =3D =
dst;</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	=
	</span>}</div><div><br class=3D""></div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre">		=
</span>struct ifaddr *ifa =3D ifaof_ifpforaddr(gw_sa, =
nh-&gt;nh_ifp);</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">		</span>if (ifa =3D=3D NULL) =
{</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">		=
	</span>NL_LOG(LOG_DEBUG, "Unable to determine ifa, =
skipping");</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">			</span>*perror =3D =
EINVAL;</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">			</span>return =
(NULL);</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">		</span>}</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre">		=
</span>nhop_set_src(nh, ifa);</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre">	</span>}</div><div>```</div><div>If the =
outgoing interface `<span style=3D"caret-color: rgb(0, 0, 0); color: =
rgb(0, 0, 0);" class=3D"">nh-&gt;nh_ifp` does not have any IPv4 =
addresses,&nbsp;</span>we should try ifaddrs on other =
interfaces,</div><div>and have 0.0.0.0 as the last sort ( in case the =
net stack is IPv6 only, i.e., no IPv4 addresses available =
)</div><div><br class=3D""></div></div><br class=3D""><blockquote =
type=3D"cite" class=3D""><div class=3D""><blockquote type=3D"cite" =
class=3D"" style=3D"font-family: Helvetica; font-size: 13px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><div class=3D""><div class=3D"" =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
13px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;"><br =
class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div =
class=3D"" style=3D"font-family: Arial, sans-serif; font-size: =
14px;">```</div><div class=3D"" style=3D"font-family: Arial, sans-serif; =
font-size: 14px;"><br class=3D""></div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;">ping6 to =
`<span class=3D"" style=3D"text-decoration: none; background-color: =
rgb(255, 255, 255); display: inline =
!important;">fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing =
wrong? Configuration of the interface is the following:</span></div><div =
class=3D"" style=3D"font-family: Arial, sans-serif; font-size: =
14px;"><span class=3D"" style=3D"text-decoration: none; =
background-color: rgb(255, 255, 255); display: inline !important;"><br =
class=3D""></span></div><div class=3D"" style=3D"font-family: Arial, =
sans-serif; font-size: 14px;">```</div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span =
class=3D"">vlan200: =
flags=3D1008843&lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP&gt; =
metric 0 mtu 1500</span><div class=3D""><span class=3D"">&nbsp; &nbsp; =
&nbsp; &nbsp; =
options=3D4600703&lt;RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,M=
EXTPG&gt;</span></div><div class=3D""><span class=3D"">&nbsp; &nbsp; =
&nbsp; &nbsp; ether fe:d9:dd:2d:35:c5</span></div><div class=3D""><span =
class=3D"">&nbsp; &nbsp; &nbsp; &nbsp; inet6 =
fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid =
0x5</span></div><div class=3D""><span class=3D"">&nbsp; &nbsp; &nbsp; =
&nbsp; groups: vlan</span></div><div class=3D""><span class=3D"">&nbsp; =
&nbsp; &nbsp; &nbsp; vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent =
interface: iavf0</span></div><div class=3D""><span class=3D"">&nbsp; =
&nbsp; &nbsp; &nbsp; media: Ethernet autoselect (10Gbase-SR =
&lt;full-duplex&gt;)</span></div><div class=3D""><span class=3D"">&nbsp; =
&nbsp; &nbsp; &nbsp; status: active</span></div><span class=3D"">&nbsp; =
&nbsp; &nbsp; &nbsp; nd6 =
options=3D23&lt;PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL</span><br =
class=3D""></div><div class=3D"" style=3D"font-family: Arial, =
sans-serif; font-size: 14px;">```</div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span =
class=3D""><br class=3D""></span></div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span =
class=3D""><br class=3D""></span></div><div class=3D"" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><br =
class=3D""></div><div class=3D"protonmail_signature_block" =
style=3D"font-family: Arial, sans-serif; font-size: 14px;"><div =
class=3D"protonmail_signature_block-user"><div class=3D"" =
style=3D"font-style: normal; font-weight: normal; letter-spacing: =
normal; text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; text-decoration: none; font-family: Helvetica; =
font-size: =
12px;">Beno=C3=AEt&nbsp;</div></div></div></div></blockquote></div><br =
class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; =
font-size: 13px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><div class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 13px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><div class=3D"">Best regards,</div><div =
class=3D"">Zhenlei</div></div></div></blockquote></div></blockquote></div>=
<br class=3D""><div class=3D"">
<div><br class=3D""></div>

</div>
<br class=3D""></body></html>=

--Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BE4D70C1-885D-4C6B-AE8C-AC62BD450F8A>