From owner-freebsd-current Mon Jun 3 14:34:13 2002 Delivered-To: freebsd-current@freebsd.org Received: from mta5.snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by hub.freebsd.org (Postfix) with ESMTP id 4A42A37B400; Mon, 3 Jun 2002 14:34:07 -0700 (PDT) Received: from nate.cryptography.com ([63.195.111.154]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GX500E4LFWUPR@mta5.snfc21.pbi.net>; Mon, 03 Jun 2002 14:34:07 -0700 (PDT) Date: Mon, 03 Jun 2002 14:34:50 -0700 From: Nate Lawson Subject: AIO test results X-Sender: nate@cryptography.securesites.com To: current@freebsd.org, stable@freebsd.org Message-id: <5.1.0.14.2.20020603143427.025b6810@cryptography.securesites.com> MIME-version: 1.0 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I have been working on several projects using AIO to break up latency in large, sequential reads. In my code, I was surprised at what appear to be performance problems where splitting a 256k read into 4 64k reads was actually quite a bit slower than a single 256k read (in fact, 3 times slower). So I cooked up this little bit of demo code called aioex (linked below) to exercise AIO and test different configurations. I've included the test results in the tgz also. Please ignore the absolute latency of some of the responses since that is a factor of the drive itself (especially the ~10 ms head seeks). I'm more concerned with the average latency of the subsequent completion handlers. In the best tests, 4 64k reads took 15-17 ms total while 1 256k read took 5-7 ms. Since the reads are all sequential, I would expect read-ahead would begin to improve things as the test went on but it didn't. With "dd bs=64k if=/mnt/testvol of=/dev/null", the drive gets around 45 MB/sec sustained which works out to around 1.5 ms per 64k block which fits perfectly with the results for AIO reads of 256k. In the test results, "prio" means I used rtprio to give aioex realtime priority (all other processes had normal priority). I also ran the test with max_aio_procs set to 1 aiod and 4 aiods. Finally, I tried it with different chunk sizes (i.e. 2 requests of 128k each). All timing was done with RDTSC. Be sure to update CYCLES_PER_SEC for your CPU in aioex.c and enable "options VFS_AIO" in your kernel. The script "runit" will help in sending back results since it prints your config as well as the results. My initial tests seem to show that giving aioex realtime priority helps a small amount and limiting it to 1 aiod helps a bit too. (I'm guessing the latter is due to the potential for requests to complete out of order with multiple aiods and the current scheduler). I'd really appreciate it if anyone could check my results and advise. I've run this past Alan Cox but he is currently busy. The tests were done on 4-stable, Celeron 500mhz, 128M ram, Adaptec 2940U2W, Quantum Atlas 10k3 drive. I use -current as well and would be interested in others' results on that platform, hence the cc. http://www.root.org/~nate/aioex.tgz Thanks, Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message