From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 1 17:05:16 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D1855E0; Sat, 1 Mar 2014 17:05:16 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0364B1AEA; Sat, 1 Mar 2014 17:05:15 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id x13so2105689qcv.5 for ; Sat, 01 Mar 2014 09:05:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UKD0jDEu5cCttp1btYnIRgYeaCfAhV3PII+xMt9Vajw=; b=0+LLqBlPljOV5KRGRUEaKssstXfQ7Bt26m9YbiATlk5zUxAR6Bw1qmF3fgXn8lKZ0v 2BUYvlaPufyvZH8BPfGmdZfKh9RkvvC4jdCe35Bp+19pxCODhjWuvaCr2GmvXFzPXcDj brPEdAIcOrGy/Qt3dECD8Ih1QC3JpFbrbDzM5ao2PxzqloI/+dRWAvZBOaTUF8TjFKpe YtsbZyDIpbmYBqblPV37BqHOt4IzBUp1t/edb/PZu7c9ELZNu0wTAzDI52zDEt4P57aL OR1324CS9vR57VEvl98q6+3GD8JUPuPOIOvv9TTwBqmYX3ljFujUNh8ieO5JJnjad39A 0EZw== MIME-Version: 1.0 X-Received: by 10.140.42.54 with SMTP id b51mr11420271qga.87.1393693515192; Sat, 01 Mar 2014 09:05:15 -0800 (PST) Received: by 10.224.16.10 with HTTP; Sat, 1 Mar 2014 09:05:15 -0800 (PST) In-Reply-To: <53120EE8.1080600@bakulin.de> References: <20140216111153.GA74858@olymp.kibab.com> <5C2CF572-360D-4CA0-81C7-18A5C455AED5@bsdimp.com> <20140224142642.GA32538@olymp.kibab.com> <53120EE8.1080600@bakulin.de> Date: Sat, 1 Mar 2014 09:05:15 -0800 Message-ID: Subject: Re: MMC/SDIO stack under CAM From: Adrian Chadd To: Ilya Bakulin Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sat, 01 Mar 2014 17:19:33 +0000 Cc: "freebsd-hackers@freebsd.org" , Alexander Motin , "freebsd-arm@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Mar 2014 17:05:16 -0000 On 1 March 2014 08:46, Ilya Bakulin wrote: > Hi Adrian, > > On 24.02.14, 16:59, Adrian Chadd wrote: >> hi, >> >> Let me just reiterate some .. well, experience doing this stuff at QCA. >> >> You really, absolutely don't want too much overhead in the MMC/SDIO >> path between whatever is issuing things and the network driver. >> >> There was significant performance work done at QCA on a local MMC/SDIO >> driver and bus to get extremely low latency and CPU utilisation when >> pushing around small transactions. The current CAM locking model is >> not geared towards getting to high transaction rates. > > So here you mean some work done on Linux MMC/SDIO stack by QCA > which made it far better than current Linux MMC stack in terms of > high SDIO I/O rates? Yup. The stock MMC stack/driver in Linux wasn't "fast" enough at small transactions to sustain the wifi speeds customers required. >> >> You may think this is a very architecturally pretty solution and it >> indeed may be. But if it doesn't perform as well as the existing local >> hacks that vendors have done, no company deploying this hardware is >> going to want to use it. They'll end up realising there's this massive >> CAM storage layer in between and either have to sit down to rip it up >> and replace it with something lightweight, or they'll say "screw it" >> and go back to the vendor supplied hacked up Linux solution. > > I think that if the "architecturally pretty solution" behaves worse than > some ugly hacks, then it may be not so pretty or the architecture is > just broken > by design. > >> So I highly recommend you profile things - and profile things with >> lots of small transactions. If the CAM overhead is more than a tiny, >> tiny fraction of CPU at 25,000 pps, your solution won't scale. :-) > > I don't really know what to compare with. For MMC/SD cards it is pretty > obvious, but then these cards will be likely the bottleneck, not the stack. > And the only goal would be to not make the stack slower than it is now. > But, as ATA devices are much faster than MMC/SD, I don't think this will > be a problem. > > For SDIO things are different. But we don't have any drivers (yet), except > mv_sdiowl that I'm writing, to test on. So I have to bring the SDIO > stack on CAM, > than bring mv_sdiowl to the state when it can actually transmit the > data, and then > compare performance with the vendor-supplied Linux driver. > We'll see then if there is a room for improvement... That sounds like a plan. Just note that although storage looks like it's doing much more throughput, the IO size also matters. As I said above, it's not uncommon to have > 1000 receive frames a second on 802.11n; and that can peak much higher than that. That's not the kind of IO rate you see on SD cards. :-) -a