From owner-freebsd-scsi@FreeBSD.ORG Fri Jul 18 10:53:48 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2DBD6A50 for ; Fri, 18 Jul 2014 10:53:48 +0000 (UTC) Received: from exprod7og117.obsmtp.com (exprod7og117.obsmtp.com [64.18.2.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90EBA253E for ; Fri, 18 Jul 2014 10:53:46 +0000 (UTC) Received: from mail-lb0-f175.google.com ([209.85.217.175]) (using TLSv1) by exprod7ob117.postini.com ([64.18.6.12]) with SMTP ID DSNKU8j8uiaDiX7ltGZD6E2h1cLKW5DHYk+6@postini.com; Fri, 18 Jul 2014 03:53:47 PDT Received: by mail-lb0-f175.google.com with SMTP id n15so2737991lbi.34 for ; Fri, 18 Jul 2014 03:53:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc:content-type; bh=QaqSp+RfsKJdiMTTk91b3Z97cILFnB0of5bptONGAiY=; b=SylFidVn5F/nIlPsDZVRSmhA4si681IcC4QbBav1PwF6mBZgq1wO1HCEyo47mSIhqp DyM4Ygq4pymiiJlu5kYu4Fl5Kw7l7dmJL+8R1BB78cKFbsIOxWIRpaS4Y4ePpYCULCai Pv0l6MCPGd9w+FJ9NJC7lCKqdpC12UGGIOdoJMTpMQtOihyLHYMZ8ugON517S1lMx7qQ yhYfhaUHUNZcXta1w17FNln63KV+LncUpC4hgyQFENrwJjGxHHBeRJItfcu1c/fbGhYp QUAQW4j50fPV//XUyZfhF77t/sassgFK4PctWar4l4KU0XWYXzOxe3/8mEpwX/E4P1jo 54uQ== X-Received: by 10.112.30.99 with SMTP id r3mr4037366lbh.14.1405680481389; Fri, 18 Jul 2014 03:48:01 -0700 (PDT) X-Gm-Message-State: ALoCoQlgbf56LaP9AsPXtsqGzHUKA/TsQtmb0wwvkiGMGikJEmjAeqW0qRoV1ZOrb9kYTm3Bx1vF3UwrsZUbJVrbH9qBPwX4x7sFBhYqupiSUI6WZDkjMosuFR2MjsoyzAW/TWLgU9yumKlqVL7bhkH9QFB4ilxwdQ== X-Received: by 10.112.30.99 with SMTP id r3mr4037359lbh.14.1405680481311; Fri, 18 Jul 2014 03:48:01 -0700 (PDT) From: Kashyap Desai References: <8fbe38cdad1e66717a9de7fdf63812c2@mail.gmail.com> <53BE8784.8060503@FreeBSD.org> <9f138f242e278476e5c542d695e58bc8@mail.gmail.com> <53BF1E6C.5030806@FreeBSD.org> <53C3A195.4020400@FreeBSD.org> In-Reply-To: <53C3A195.4020400@FreeBSD.org> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQG8AScvFWC5/cNVmZuoGGC8NZGkUAIYsfkbAp/trHgBd5fqmAJDItauAvNNSe6bcbNDIA== Date: Fri, 18 Jul 2014 16:18:00 +0530 Message-ID: Subject: RE: SSDs peformance on head/freebsd-10 stable using FIO To: Alexander Motin Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD-scsi X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 10:53:48 -0000 > -----Original Message----- > From: Alexander Motin [mailto:mavbsd@gmail.com] On Behalf Of Alexander > Motin > Sent: Monday, July 14, 2014 2:54 PM > To: Kashyap Desai > Cc: FreeBSD-scsi > Subject: Re: SSDs peformance on head/freebsd-10 stable using FIO > > On 14.07.2014 11:36, Kashyap Desai wrote: > > From: Alexander Motin [mailto:mavbsd@gmail.com] On Behalf Of > Alexander > >> First thing I noticed in this profile output is bunch of TLB shutdowns. > >> You can not reach reasonable performance from user-level without > >> having HBA support unmapped I/O. Both mps and mpr drivers support it, > >> but for some reason still not mrsas. Even at non-peak I/O rates on > >> multi-core system TLB shutdowns in such case can eat additional 30% > >> of CPU time. > > > > Thanks.! For this part, I can try In mrsas. Can you help me to > > understand what you mean by unmapped I/O ? > > That is a capability to work with data not mapped into the kernel virtual > address space, i.e. to work with physical addresses instead of virtual. > Main > prerequisite to support that is that driver should not try to access the > transferred data (because it can't do it for addresses not mapped to KVA). > If > that is true, then usually only minor modification is needed to teach the > driver to receive physical addresses from CAM. > > Looking on mps driver as example you may see PIM_UNMAPPED flag > reporting unmapped I/O support to CAM, and bus_dmamap_load_ccb() > helper function transparently doing all the physical address handling > magic. Thanks Motin.. I got the what is an issue here about UNMAPPED IO. Konstantin Belousov posted below performance result and there was also a changes in to use Unmapped IO. https://kib.kiev.ua/kib/pgsql_perf.pdf I will work on this (will refer mpt driver ) and get back to FreeBSD Developers to seek any help.. ` Kashyap > > -- > Alexander Motin