Date: Fri, 05 Feb 2021 05:31:59 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 253261] AWS EC2 disk I/O performance is extremely bad Message-ID: <bug-253261-227@https.bugs.freebsd.org/bugzilla/>
index | next in thread | raw e-mail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253261 Bug ID: 253261 Summary: AWS EC2 disk I/O performance is extremely bad Product: Base System Version: 12.2-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Many People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: raj@gusw.net I am trying to move all my AWS systems back from Amazon Linux to FreeBSD. But I encounter massive slowness. People have reported about slow ENA network driver before. Even worse is the performance of the NVMe nvd disks. This is so bad that my attempt to go with FreeBSD is now completely dead. And it makes me really sad. FreeBSD was once "the power to serve" and now it feels like I'm running Ultrix on a VAX for the fun of it but for any real work I need Linux. So sad! The issue can be reproduced by anyone on AWS. Just get a FreeBSD 12 (latest) instance and allocate a gp2 of 1 TB or a gp3 of smaller size for which you specify 3000 IOPS and 250 MB/s IO capacity. Let's say that disk is /dev/nvd2 Now do this: dd if=/dev/zero of=/dev/nvd2 bs=100M status=progress and you see that it's writing with the whopping speed of 70 MB/s. That used to be good, but it is no longer good. Compare Amazon Linux doing the same thing at 300 MB/s. Now, when you put a file system over it, zfs or ufs, then instantly the performance gets better: newfs /dev/nvd2 mount /dev/nvd2 /mnt dd if=/dev/zero of=/mnt/test bs=100M status=progress now that works at about 250 MB/s. Decent. So, problem solved? No! It turns out if I create a PostgreSQL database over this setup, then again there is massive delay on the read and write and throughput will drop to even worse than 70 MB/s. Creating one index takes 10 times as long as that same on the Linux system. The dd issue is probably the best manifestation for fixing at least part of the problem. Is this about tuning? Even if it was, there is a severe problem with how the system is set up by default. But I'm afraid it's not even about tuning so bad is it. -- You are receiving this mail because: You are the assignee for the bug.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-253261-227>