From owner-freebsd-scsi@FreeBSD.ORG Mon Jul 14 09:23:40 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A892A19A for ; Mon, 14 Jul 2014 09:23:40 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DBFB222E for ; Mon, 14 Jul 2014 09:23:40 +0000 (UTC) Received: by mail-wi0-f177.google.com with SMTP id ho1so2158362wib.10 for ; Mon, 14 Jul 2014 02:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=th7CLfNtDUwHZEWYrPY5GDhpDnZa8y4bH/6fHIkW/j0=; b=qv/Bx8zB4AAL3yNxtlqcBVWzO1ElYkVlI6KZ3U3puP3v1zqxKNwGqGwAYG00X9hWWF rjot6hnGFcM6swNUDulzIePMAL6/B8mAuv8EO1csEyZm9cAhYdGL8wSFiuE2J1kEr42Q QxnmgMO2wPqAn3YRg+ZwTKPA4XSw+Nn0Ysuas8kzjJKZ5T8Po28KsNrOVlTbwuxShQml w32JIqnb+Kt9TOi1Ccawq3H0M3nR1bkmgIidmQ5Knz2Gphbpma1PK1GU1nu7pf1U1snk gqd/4FWiOOlCvLQ5lRAhLFR0W5dc2wjJaxQtPL/2j5gNIWl6LYbuv2A7N3XXloxx3r1H tC3w== X-Received: by 10.180.91.225 with SMTP id ch1mr23518473wib.34.1405329816737; Mon, 14 Jul 2014 02:23:36 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id w10sm28931192wie.22.2014.07.14.02.23.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Jul 2014 02:23:35 -0700 (PDT) Sender: Alexander Motin Message-ID: <53C3A195.4020400@FreeBSD.org> Date: Mon, 14 Jul 2014 12:23:33 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Kashyap Desai Subject: Re: SSDs peformance on head/freebsd-10 stable using FIO References: <8fbe38cdad1e66717a9de7fdf63812c2@mail.gmail.com> <53BE8784.8060503@FreeBSD.org> <9f138f242e278476e5c542d695e58bc8@mail.gmail.com> <53BF1E6C.5030806@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: FreeBSD-scsi X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2014 09:23:40 -0000 On 14.07.2014 11:36, Kashyap Desai wrote: > From: Alexander Motin [mailto:mavbsd@gmail.com] On Behalf Of Alexander >> First thing I noticed in this profile output is bunch of TLB shutdowns. >> You can not reach reasonable performance from user-level without having >> HBA support unmapped I/O. Both mps and mpr drivers support it, but for >> some reason still not mrsas. Even at non-peak I/O rates on multi-core >> system >> TLB shutdowns in such case can eat additional 30% of CPU time. > > Thanks.! For this part, I can try In mrsas. Can you help me to understand > what you mean by unmapped I/O ? That is a capability to work with data not mapped into the kernel virtual address space, i.e. to work with physical addresses instead of virtual. Main prerequisite to support that is that driver should not try to access the transferred data (because it can't do it for addresses not mapped to KVA). If that is true, then usually only minor modification is needed to teach the driver to receive physical addresses from CAM. Looking on mps driver as example you may see PIM_UNMAPPED flag reporting unmapped I/O support to CAM, and bus_dmamap_load_ccb() helper function transparently doing all the physical address handling magic. -- Alexander Motin