Date: Wed, 04 Sep 2013 12:00:39 +0300 From: Alexander Motin <mav@FreeBSD.org> To: Johan Hendriks <joh.hendriks@gmail.com> Cc: FreeBSD <freebsd-current@freebsd.org> Subject: Re: [RFC][CFT] GEOM direct dispatch and fine-grained CAM locking Message-ID: <5226F6B7.3070706@FreeBSD.org> In-Reply-To: <5226ED39.3070200@gmail.com> References: <520D4ADB.50209@FreeBSD.org> <5224511D.4090503@FreeBSD.org> <5226ED39.3070200@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 04.09.2013 11:20, Johan Hendriks wrote: > Alexander Motin wrote: >> I would like to invite more people to review and test my patches for >> improving CAM and GEOM scalability, that for last six months you could >> see developing in project/camlock SVN branch. Full diff of that branch >> against present head (r255131) can be found here: >> http://people.freebsd.org/~mav/camlock_patches/camlock_20130902.patch >> >> Heavy CAM changes there were focused on reducing scope of SIM lock to >> only protecting SIM internals, but not CAM core. That allows many >> times reduce lock congestion, especially on heavily parallel request >> submission with GEOM changes below. More detailed description of >> changes you could see here earlier: >> http://docs.freebsd.org/cgi/mid.cgi?520D4ADB.50209 >> >> GEOM changes were focused on avoiding switching to GEOM up/down >> threads in relatively simple setups where respective classes don't >> require it (and were explicitly marked so). That allows save on >> context switches and on systems with several HBAs and disks talk to >> them concurrently (that is where CAM locking changes are handy). Such >> classes were modified to support it: DEV, DISK, LABEL, MULTIPATH, NOP, >> PART, RAID (partially), STRIPE, ZERO, VFS, ZFS::VDEV, ZFS::ZVOL and >> some others. Requests to/from other classes will be queued to GEOM >> threads same as before. >> >> Together that allows to double block subsystem performance on high (at >> least 100-200K) IOPS benchmarks, allowing to reach up to a million >> total IOPS, while keeping full compatibility with all major ABIs/KBIs. >> >> Since we are already in 10.0 release process and changes are quite >> big, my plan is to wait and commit them to head branch after the >> freeze end, and then merge to stable/10. I hope the release process >> will go on schedule to not delay this work for another six months. >> >> This work is sponsored by iXsystems, Inc. >> > Hello i would like to test this patch set. > But how can i stress the machine, do you have a script or something i > can use to make the system do heavy I/O on the disks! For testing IOPS over RAW disks or different GEOM providers (to exclude FS influence) I am using such a trivial synthetic test: #!/bin/sh for disk in da0 da1 da2 da3 da4 da5 da6 da7 da8 da9 da10 da11 da12 da13 da14 da15 do for i in `jot 16` do dd if=/dev/$disk of=/dev/null bs=512 & done done iostat -zxw3 -c12 | grep total |tail -n 10 |cut -c 6-18 killall dd For total IOPS measurement in above script I am using dirtily hacked iostat tool that prints summary values in addition to per-disk ones: http://people.freebsd.org/~mav/camlock_patches/iostat.patch BTW I've uploaded new patch with some more minor CAM changes: http://people.freebsd.org/~mav/camlock_patches/camlock_20130904.patch -- Alexander Motin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5226F6B7.3070706>