Date: Mon, 24 Feb 2014 07:59:40 -0800 From: Adrian Chadd <adrian.chadd@gmail.com> To: Ilya Bakulin <ilya@bakulin.de> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, Alexander Motin <mav@freebsd.org>, "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Re: MMC/SDIO stack under CAM Message-ID: <CAJ-VmomNzCMc1T=0jAnyd_uGXbvgeTzZTtmhUPSvZ0DKUEjtKg@mail.gmail.com> In-Reply-To: <20140224142642.GA32538@olymp.kibab.com> References: <20140216111153.GA74858@olymp.kibab.com> <5C2CF572-360D-4CA0-81C7-18A5C455AED5@bsdimp.com> <20140224142642.GA32538@olymp.kibab.com>
next in thread | previous in thread | raw e-mail | index | archive | help
hi, Let me just reiterate some .. well, experience doing this stuff at QCA. You really, absolutely don't want too much overhead in the MMC/SDIO path between whatever is issuing things and the network driver. There was significant performance work done at QCA on a local MMC/SDIO driver and bus to get extremely low latency and CPU utilisation when pushing around small transactions. The current CAM locking model is not geared towards getting to high transaction rates. You may think this is a very architecturally pretty solution and it indeed may be. But if it doesn't perform as well as the existing local hacks that vendors have done, no company deploying this hardware is going to want to use it. They'll end up realising there's this massive CAM storage layer in between and either have to sit down to rip it up and replace it with something lightweight, or they'll say "screw it" and go back to the vendor supplied hacked up Linux solution. So I highly recommend you profile things - and profile things with lots of small transactions. If the CAM overhead is more than a tiny, tiny fraction of CPU at 25,000 pps, your solution won't scale. :-) -a On 24 February 2014 06:26, Ilya Bakulin <ilya@bakulin.de> wrote: > On Sun, Feb 16, 2014 at 11:05:35AM -0700, Warner Losh wrote: >> that's cool! I'd thought of writing a mmcsim that I could use to develop= the stack on x86, but time pressures of my job precluded that (though in h= ind sight, it would have saved a lot of time). > > I assume that by "mmcsim" you mean not MMC SIM for CAM, but MMC SIMulator= :-) > Yeah so essentially it should be able to return some "fake" data > to the upper layer, maybe based on configuration (ie 1 GB SD, 16 GB SDHC,= ...). > >> > For SDIO card, the situation will be different. Essentially, >> > we should allow a device driver to send arbitrary messages to the card= . >> > So the device driver will attach directly to the scbus. >> > >> > Please let me know your thoughts about it. >> > I really want SDIO make its way into the kernel, because there >> > is an increasing number of ARM boards on the market that have integrat= ed >> > SDIO WLAN on them and we cannot support them fully. >> >> Generally, I like this idea... I'd be very interested in some of the sp= ecifics to make the direct attachment work with SDIO. That's the one area I= either don't think I understand your proposal well enough, or I do and I'm= concerned about it. :) So maybe write a bit more about the details of the = SDIO cards and how they's interact with CAM in this scenario... >> >> The notion of moving MMC/SD into the CAM stack has been around since I s= tarted working on MMC. At the time, CAM was too SCSI centric to do it, but = Alexander Motin's work has generally improved that situation, so it may be = possible now to do it and shake out the few dark corners of CAM where this = isn't the case. I also think it should help performance a lot since we're c= urrently single threaded to the card (but that's also the nature of the bus= ), which introduces some round-trip latencies between too many threads that= CAM may help us avoid. >> >> Warner > > After talking with Alexander about CAM, here is the updated proposal: > > * All card communication logic should be placed in MMC XPT layer, with th= e new CCB type > that describes MMC request + some data needed by the system. > So almost everything from sys/dev/mmc/mmc.c goes there. > * da(4) is a driver for SCSI disks. It makes more sense to adapt mmcsd(4)= to use CAM. > So mmcsd(4) will create CCBs and pass them to CAM layer, getting async = notifications > via standard CAM callback mechanism. > * It is possible to write drivers for SDIO devices exactly like adopted m= mcsd(4). > Every such driver would be then built as CAM peripheral driver, which m= eans it will be able > to send/receive CCBs. > To do that, every driver has to call PERIPHDRIVER_DECLARE() macro, subs= cribing to CAM "new device found" events. > > The ongoing work for MMC CAM is here: > https://github.com/kibab/freebsd/compare/master...mmccam > Not so much stuff here yet. > > PS: In theory, ssince there is already an existing interface for passing = CAM CCBs between userland and kernel, > it WOULD be posssible to create device drivers entirely in userland... Li= ke microkernel OSes do. > But our network stack doesn'have such feature I guess... > > -- > Ilya
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmomNzCMc1T=0jAnyd_uGXbvgeTzZTtmhUPSvZ0DKUEjtKg>