Date: Fri, 6 Nov 2015 13:10:10 +0100 From: Zbigniew Bodek <zbb@semihalf.com> To: Andrew Turner <andrew@fubar.geek.nz> Cc: "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Re: HEADS UP: Cavium ThunderX support in the tree Message-ID: <CAG7dG%2BwN-%2BT2SggMy1Of0h3X613owqbcTD%2BnUcKdV_C=ggOfMA@mail.gmail.com> In-Reply-To: <20151106113641.02e0f826@bender> References: <CAG7dG%2BxzC_SGiFtncTd7tbi_VqmyXj_08JY34hKFDkvnSg1PPw@mail.gmail.com> <20151106094714.5e8632c6@bender.Home> <CAG7dG%2ByvGq=-vRo3swCGD_goo6UhQy3CW66GUbikUVOvVYkA6Q@mail.gmail.com> <20151106113641.02e0f826@bender>
next in thread | previous in thread | raw e-mail | index | archive | help
2015-11-06 12:36 GMT+01:00 Andrew Turner <andrew@fubar.geek.nz>: > On Fri, 6 Nov 2015 12:23:11 +0100 > Zbigniew Bodek <zbb@semihalf.com> wrote: > >> 2015-11-06 10:47 GMT+01:00 Andrew Turner <andrew@fubar.geek.nz>: >> > On Thu, 5 Nov 2015 17:14:03 +0100 >> > Zbigniew Bodek <zbb@semihalf.com> wrote: >> > >> >> Hello all, >> >> >> >> Semihalf is happy to officially announce that starting from SVN >> >> revision r289550 FreeBSD is ready to run on Cavium ThunderX >> >> system-on-chip! >> >> ThunderX is the first ARM64 (ARMv8) hardware platform to be >> >> supported by FreeBSD and the only one that introduces 96-CPU cores >> >> SMP. The integrated code includes support for: >> >> >> > ... >> >> - Network Interface Controller (VNIC) 1/10/40G >> > >> > We do? Then why don't I see it in ifconfig on the ThunderX in the >> > cluster? >> > >> > root@cavium:~ # ifconfig -a >> > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu >> > 1500 >> > options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO> >> > ether 00:15:17:0b:6b:08 >> > inet 192.168.5.167 netmask 0xffffff00 broadcast >> > 192.168.5.255 media: Ethernet autoselect (1000baseT <full-duplex>) >> > status: active >> > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 >> > options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> >> > inet6 ::1 prefixlen 128 >> > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 >> > inet 127.0.0.1 netmask 0xff000000 >> > groups: lo >> > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> >> > >> > Andrew >> >> Strange. Works for me ;P >> >> First of all, networking cards on ThunderX are PCI Virtual Functions >> so you need to bring them up using iovctl. >> If you watched the youtube demo you should have seen that at first >> there are no interfaces in ifconfig and then I do iovctl and they >> appear. > > Ok, so it's similar to wlan where we need some extra configuration > before we can use it. What is the option I need to enable in rc.conf to > enable this? Yes. You need to invoke iovctl with proper parameters and configuration file. https://www.freebsd.org/cgi/man.cgi?query=iovctl.conf&sektion=5&apropos=0&manpath=FreeBSD+11-current > >> Secondly, EFI that runs on the board needs to pass the correct DTB to >> the kernel (it has to have PCIB, BGX, MDIO, PHYs nodes and they need >> to be in the correct order [MDIO needs to attach before BGX]). > If this is the case the code is broken. It should just work with > the vendor dtb and GENERIC. Thanks for quick evaluation but please see comment below. > > We already have to handle this in other parts of the code by having the > dependent drivers on an earlier pass than later drivers. MDIO is a platform device whereas BGX, NICPF (physical function) and NICVF (virtual function) are devices on PCI. Therefore if the PCIB is placed prior to MDIO in DTB all devices on this PCI bridge will be attached before MDIO is attached. As far as I know there is no straightforward way to create a logical relation between device on PCI and platform devices enumerated using DTB. That is why we asked Cavium to change the order in their DTBs. Of course if you know how to workaround that I would be happy to apply this change to "unbreak" the existing code. > > I also don't see any documentation. I can see the bgx driver, bit there > doesn't seem to be a manual page. > root@cavium:~ # man bgx > No manual entry for bgx > > Andrew Correct. There is no manual entry for: BGX, Cavium MDIO, NICPF and NICVF. Best regards zbb
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG7dG%2BwN-%2BT2SggMy1Of0h3X613owqbcTD%2BnUcKdV_C=ggOfMA>