From owner-freebsd-hackers@FreeBSD.ORG Sat Jul 21 02:30:17 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA09B106564A; Sat, 21 Jul 2012 02:30:16 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by mx1.freebsd.org (Postfix) with ESMTP id BCB548FC08; Sat, 21 Jul 2012 02:30:09 +0000 (UTC) Received: from [192.168.1.2] (pool-72-89-250-138.nycmny.fios.verizon.net [72.89.250.138]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 245041B404F; Sat, 21 Jul 2012 02:30:07 +0000 (UTC) Message-ID: <500A13A6.7030503@gentoo.org> Date: Fri, 20 Jul 2012 22:27:50 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.5) Gecko/20120628 Thunderbird/10.0.5 MIME-Version: 1.0 To: Adrian Chadd References: <50085193.6030203@gentoo.org> <5009DB2A.7070408@gentoo.org> In-Reply-To: <5009DB2A.7070408@gentoo.org> X-Enigmail-Version: 1.3.5 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigD9FC2196913C94EE702A8575" Cc: "hackers@FreeBSD.org" , current@freebsd.org, ivoras@freebsd.org Subject: Re: Awful FreeBSD 9 block IO performance in KVM X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jul 2012 02:30:17 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigD9FC2196913C94EE702A8575 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 07/20/2012 06:26 PM, Richard Yao wrote: > On 07/20/2012 03:44 PM, Adrian Chadd wrote: >> On 19 July 2012 11:27, Richard Yao wrote: >>> Dear Everyone, >>> >>> FreeBSD 9 has awful block IO performance in KVM. I have experienced i= t >>> and others have experienced it. Someone posted slides to slideshare w= ith >>> benchmarks documenting it: >>> >>> http://www.slideshare.net/TakeshiHasegawa1/runningfreebsdonlinuxkvm >>> >>> Slides 13 and 20 are particular eye openers. Does anyone know what is= wrong? >> >> For those watching at home - this is bad performance _with_ the virtio= >> drivers themselves, not just with SCSI emulation. >> >> Slide 17 is very telling - the operation latency is quite high. >> >> Richard, are you able to easily test out things on FreeBSD-HEAD guest >> in a Linux KVM? If so, some of the storage/block/GEOM driver people >> may be able to step up and start offering some ideas. >> >> Thanks, >> >> >> >> Adrian >=20 > Dear Adrian, >=20 > I am in the process of setting up a VM instance specifically for this. > While installing it, I noticed that qemu-kvm printed 'lsi_scsi: error: > ORDERED queue not implemented', which might be a clue as to why the > block device performance is bad. >=20 > Also, I will try testing raw disk IO for Ivan after I have it setup. >=20 > Yours truly, > Richard Yao >=20 I now have FreeBSD 9.1-BETA1 installed in a virtual machine. I noticed the following in dmesg which might explain why the emulated SCSI support is so slow: da0 at sym0 bus 0 scbus2 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 3.300MB/s transfers da0: Command Queueing enabled da0: 409600MB (838860800 512 byte sectors: 255H 63S/T 52216C) It does not explain why virtio is slow though, although I still need to test virtio against the latest code. I will do ivan's raw block test against virtio-blk, mainly because there is no point in doing it against a device whose transfers have been capped to 3.3MB/sec. --------------enigD9FC2196913C94EE702A8575 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJQChOpAAoJECDuEZm+6ExkWo4P/3O1QiiK9lu6NGfnEmduwhzo NE6Oi1I/CEYkKwhxM8sjmM01vUsmnfcS+uJ1PyT1Ncdnz91FT6AA8wM1ifM+RUdz qgzZOywLfgIDoxOdvefGejg8eFhFMae4L0FiMAZBiS3NO7S8ys1fOSCZvOAnuAAL pYihDi/v94k/tIh3vIzREQff+24DHb2c6o2ZFQDquu0H/3b6pL3/rDMKquNQ7zzG KmRR3iaGc6EZxVdez8cYB8093zmiZaUWScvbVR08DPJljBetCA5jKoR5+PcdE5FE 5W4UJizOuGY3/uabvbOGRGxL1Y2HUtU4F5QsjLGOArqBazUQr5gYX4Ha1lAaPqc3 4UNLoC+oEUZo6+ZVjM8HuVNyO/PFxap6lg3klMrj1AwE9UX1eOpUoW0817rQM/RI ApGQSz809CZe8oIJTdCBE/OParyOGMoqS9lYfl/hqVrCBFbMlxNXisdiczsJPPkT QnDvCywISWMXlhPSJDeH9EeO6+ZUK1bYPZEA4vLCWGdXJ18e085RJIjQ55Hvwira RtC5djfanTPsPk35KKyM6eqvcf+95QnK0da6NFoACF9olLXUY1UgoYF3MuWi3Rf2 QpDK1sIkwdsVT+id4UwYIWxCNhHt9QIwEOAn3jKD3QvS+AQZ6Ezu6dPwuK9Olj53 rszGbmHsQbSJCap9jGdR =XdrQ -----END PGP SIGNATURE----- --------------enigD9FC2196913C94EE702A8575--