From owner-freebsd-scsi@FreeBSD.ORG Thu Jan 29 20:17:15 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE75F786 for ; Thu, 29 Jan 2015 20:17:15 +0000 (UTC) Received: from nm19-vm1.bullet.mail.gq1.yahoo.com (nm19-vm1.bullet.mail.gq1.yahoo.com [98.136.217.24]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 97E48260 for ; Thu, 29 Jan 2015 20:17:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1422562628; bh=mx/1ih0Fd0gSjZl6GFX0YNzVTIni1gp6/p2gI1dNYHo=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject; b=LKWHArii1XA+YA2ct2eeBhXy5vJgIzeTVCEfYD/wzFgQw04d3xkirBA5Gqgn2rkvH7TkDCGP9+EQiC30tAp5J7sdGzi9TgllaL8gyDVZshADYJDhIfYUgYtMPSbs1PW4vEkYFSefHot+bbiRnMXyTgBqqgg44hYSnmb2ydsXXvVh6MSZMHeX6Y03SaLrcB0Q5BCe7sZjNiOmY3Gla3qXuwog1NM5IoN2fttPr1WvJh3YiEbANY5JxsbqG+xH4VYW9BEGQSUqkgiG+enNudr4mXrFu2PIHOvHguJUyXEKDVL2ejlgfub9IzjBGvLMutZPhbjhRvb3NvYI72r0SgFyYA== Received: from [98.137.12.55] by nm19.bullet.mail.gq1.yahoo.com with NNFMP; 29 Jan 2015 20:17:08 -0000 Received: from [208.71.42.202] by tm15.bullet.mail.gq1.yahoo.com with NNFMP; 29 Jan 2015 20:17:08 -0000 Received: from [127.0.0.1] by smtp213.mail.gq1.yahoo.com with NNFMP; 29 Jan 2015 20:17:08 -0000 X-Yahoo-Newman-Id: 286312.33753.bm@smtp213.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: ZxXRApgVM1lwAkM8fHn0tNk.UJOvi748AfHXORNdIi0tAtR p5W_q2fxlzOBj1au42bCffu5p4sxGG03qfQje3YFzp.oJSh8uuoJM.2Ce90P _2H1pdJrKO_Toy9HuFyWWuJTdq2v3NRZ9pgBN03gYoxRwZSs5wejJDoVQmSn AzAOcuF_mErjkhj2.X7dJc3rZTlanLHy8sCRcfLX1tvK.JC6u2WGX9h8wiWQ gIw8siO3LJQTQoQZ6AF3EiDAKzSMt66wrPKAs6t9VMpOC2dIZ4hDSb3yzZI7 jXeAczomIn97yW3nn8M4As6PYB62ef0gTEhK_NSEacwnrAhaBKTH3qNT4GRM 8OsUPS3CJLRcvXJCszs.ziC2AgzDTUWuCmV6V8QUqIAYcod5E_DWzSEIjLL0 9uwee3M1zYXXjI15jc2xy_oktRqTvQsn.xZDz5BC6CG06PRr.o3R6RBBzqlS 1e5.BVutpQ.7stzCPcFoiOv9zGOQrMIQrtKVGFO2VRpVqlw8AuTR4Ta08ljq KlUBbnAle38a1tiKdEmE8hERR9I2w56gkZebnYxDeJQ-- X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: How to send 1MB I/O size in a single I/O request without split From: Scott Long In-Reply-To: Date: Thu, 29 Jan 2015 13:17:06 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8B56B74C-7EBC-4D1B-89AB-46DA8ED05DD5@yahoo.com> References: To: Sibananda Sahu X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2015 20:17:16 -0000 > On Jan 29, 2015, at 11:56 AM, Sibananda Sahu = wrote: >=20 > Hi All, >=20 >=20 >=20 > Recently we have added large I/O size of 1MB in our LSI controller and = for > that we have implemented the same in driver. >=20 > But I have observed that the large I/O that an application is able to = send > is 128KB in one I/O request. >=20 >=20 >=20 > I used the following command to send 1MB I/O: >=20 > # ddpt if=3D/dev/da0 of=3D/dev/zero count=3D1 bs=3D1M >=20 >=20 >=20 > But I have observed that the number of scatter gather elements per I/O > request always comes 1 and the I/O length comes as 128KB(max). >=20 > [=E2=80=A6] >=20 >=20 > So my primary questions are: >=20 > - How can I send a large I/O size of 1MB in a single I/O = request > without any split? You answered this question already, you must redefine MAXPHYS. This can = be done via a kernel compile option, and many users and companies = already know how to do this. I plan to start a discussion on increasing = the default size. >=20 > - Why I am getting always 1 scatter gather element? >=20 You are likely getting an allocation that is physically contiguous. = This is especially true since you are using =E2=80=98dd=E2=80=99 from = userland, and the memory allocator in userland tries to useon 2MB = superpages for allocations. The busdma API will see that the allocation = is contiguous and attempt to merge the contiguous segments. This is = usually desirable since few segments reduces processing overhead in the = driver and the hardware. > - How can I get more sge count in an I/O request? >=20 If you want to test multiple segments, I suggest leaving you system = running for a long time with multiple processes freeing and allocating = memory so that the system becomes fragmented. You can also modify the = bus_dma_tag in your driver to specify a maximum segment size of 4k = instead of (presumably in your case) something larger. That will force = busdma to stop merging adjacent segments. Scott