Date: Sun, 09 Mar 2008 22:45:53 +0100 From: Rafal Jaworowski <raj@semihalf.com> To: Bill Paul <wpaul@FreeBSD.org> Cc: marcel@FreeBSD.org, cvs-src@FreeBSD.org, xcllnt@mac.com, cvs-all@FreeBSD.org, src-committers@FreeBSD.org Subject: Re: cvs commit: src/sys/powerpc/booke machdep.c Message-ID: <47D45A91.3080602@semihalf.com> In-Reply-To: <20080309202633.1DEC11065672@hub.freebsd.org> References: <20080309202633.1DEC11065672@hub.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Bill Paul wrote: >>>> BTW: currently U-Boot is broken for most of PPC variations (including >>>> 85xx) in >>>> the following way: d-cache is not being flushed before disabling and >>>> strange >>>> things happen as you imagine. For a quick workaround apply something >>>> like this >>>> when using ELF images: >>>> http://people.freebsd.org/~raj/patches/misc/uboot-dcache-workaround.diff >>> Yup. That's what we do at Juniper. I don't see any bad effects other >>> than having to invalidate the cache before (re-)enabling it so I don't >>> have that patch applied myself... >>> >> I have seen strange things, including hangs, and also TSEC non-functional >> effects due to this issue... :) >> >>> The problem I'm having with U-Boot is with the TSEC driver as you've >>> seen on the U-Boot mailing list. I have one MPC8555CDS sitting idle >>> after the U-Boot upgrade because I can't netboot :-/ >>> >> Yes, I saw. Let me re-flash my 8555 with the latest image and try, but when >> I'm not remote i.e. Monday. Don't know if you know this tip, but there used to >> be a second flash bank on the MPC8555CDS that you can access while running >> from the boot one, so you can recover in such scenario without a h/w debugger >> by copying over that backup flash bank, provided you have a working image there... >> >> Rafal >> > > I am not on the U-Boot mailing list, so I'm not familiar with the TSEC > problems you're having, but I'm curious: are you saying that when you > try to boot a FreeBSD kernel image with the cache disabled, U-Boot can > download and start the kernel, but FreeBSD fails to mount its rootfs > correctly because of problems with the TSEC? > I guess I didn't say anything like this :) There are two issues, both local to U-Boot and not affecting kernel TSEC driver operation: 1. What Marcel is suffering from is some basic problem with TSEC not working at all (link failure) after U-Boot update on one of his systems. 2. I referred to a U-Boot bug present on most PPC variations, which might surface when running standalone apps enveloped in ELF: d-cache is disabled before passing control, without flushing it first, so depending on the data which didn't get flushed there might be corruptions or undesired effects within the standalone app. One of the problems I was observing was that TSEC stopped working after return from a standalone app to U-Boot, also other corrupt data happend, hangs etc. FreeBSD loader(8) for U-Boot environment is such an external app and when this issue hit the loader wouldn't usually be able to load the kernel via network at all. The pure binary produced from this same ELF image works without ill effects ('go' command doesn't do cache trickery before passing control). Please note the workaround patch for handling ELF images does not disable cache, just the opposite: it prevents from doing so. > If that is what you're saying, then something about this is very confusing: > the FreeBSD TSEC driver appears to be enabling the TSEC's RX and TX snooping > featurss, so in theory (if I understand things correctly, which I might not), > the cache behavior really shouldn't affect it. With VxWorks, the bootrom > typically runs with the data cache disabled and then the VxWorks image > enables it once it's loaded, but the same TSEC driver works properly in > both scenarios. > Our TSEC driver works fine with caches enabled. > Some questions that I have: > > - Is bus_dma aware of the fact that the TSEC is snooped? Because if it is, > it seems to me that all the sync operations should reduce to no-ops. > _bus_dmamap_sync() for PowerPC is a no-op. TSEC driver uses DMA sync ops per convention (it's also possible it could be integrated with something else than PowerPC core one day, so we are prepared :) > - In the encap routine, the driver always sets the R bit immediately on the > first descriptor of every outbound frame. (The Ready bit is basically > the ownership bit that indicates to the TSEC that the descriptor is > ready for processing.) The VxWorks driver always leaves the R bit in > first descriptor cleared until all subsequent descriptors have been > set up. This is to prevent the TSEC from potentially racing with the > encap routine and processing the first few descriptors of an outbound > frame before all descriptors are ready. It might be a good idea to have > the encap routine in the FreeBSD driver work the same way. I know the > idea is to set up all the descriptors first and then initiate transmission > by setting the THLT bit in the TSTAT register, but this doesn't account > for the possibility that the transmitter is already busy with a previous > transmit request when you go to set up new descriptors. > Thanks for pointing this out. I'll have a look; there are more changes and improvements to TSEC in my queue, hope you don't mind reviewing these too? > - Has anyone actually tried to debug the TSEC issue, such as by setting up > tcpdump to capture traffic from the 8555 or instrumenting the driver? > It might be worthwhile to actually find the problem rather than just > saying "it works if we hack U-Boot" since this might be a symptom of > a larger problem that needs to be fixed. > As above, the hack is preventing from some other problems, that need to be addressed in U-Boot code, and not quite related to the Ethernet controller and its driver. Rafal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47D45A91.3080602>