Date: Thu, 29 Jan 2015 13:17:06 -0700 From: Scott Long <scott4long@yahoo.com> To: Sibananda Sahu <sibananda.sahu@avagotech.com> Cc: freebsd-scsi@freebsd.org Subject: Re: How to send 1MB I/O size in a single I/O request without split Message-ID: <8B56B74C-7EBC-4D1B-89AB-46DA8ED05DD5@yahoo.com> In-Reply-To: <f94a31843fde43237d9aa13bbe543ddf@mail.gmail.com> References: <f94a31843fde43237d9aa13bbe543ddf@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Jan 29, 2015, at 11:56 AM, Sibananda Sahu = <sibananda.sahu@avagotech.com> wrote: >=20 > Hi All, >=20 >=20 >=20 > Recently we have added large I/O size of 1MB in our LSI controller and = for > that we have implemented the same in driver. >=20 > But I have observed that the large I/O that an application is able to = send > is 128KB in one I/O request. >=20 >=20 >=20 > I used the following command to send 1MB I/O: >=20 > # ddpt if=3D/dev/da0 of=3D/dev/zero count=3D1 bs=3D1M >=20 >=20 >=20 > But I have observed that the number of scatter gather elements per I/O > request always comes 1 and the I/O length comes as 128KB(max). >=20 > [=E2=80=A6] >=20 >=20 > So my primary questions are: >=20 > - How can I send a large I/O size of 1MB in a single I/O = request > without any split? You answered this question already, you must redefine MAXPHYS. This can = be done via a kernel compile option, and many users and companies = already know how to do this. I plan to start a discussion on increasing = the default size. >=20 > - Why I am getting always 1 scatter gather element? >=20 You are likely getting an allocation that is physically contiguous. = This is especially true since you are using =E2=80=98dd=E2=80=99 from = userland, and the memory allocator in userland tries to useon 2MB = superpages for allocations. The busdma API will see that the allocation = is contiguous and attempt to merge the contiguous segments. This is = usually desirable since few segments reduces processing overhead in the = driver and the hardware. > - How can I get more sge count in an I/O request? >=20 If you want to test multiple segments, I suggest leaving you system = running for a long time with multiple processes freeing and allocating = memory so that the system becomes fragmented. You can also modify the = bus_dma_tag in your driver to specify a maximum segment size of 4k = instead of (presumably in your case) something larger. That will force = busdma to stop merging adjacent segments. Scott
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8B56B74C-7EBC-4D1B-89AB-46DA8ED05DD5>