Date: Tue, 22 Feb 2022 12:51:41 -0700 From: Warner Losh <imp@bsdimp.com> To: Pete French <pete@twisted.org.uk> Cc: FreeBSD Stable Mailing List <freebsd-stable@freebsd.org> Subject: Re: Instance drives in AWS comming up with the wrong size Message-ID: <CANCZdfrye3mBj5Ej=vjoUUtKx%2BqNX%2BXBXq15j6B0FCnZDZw54Q@mail.gmail.com> In-Reply-To: <b6fae531-edaf-d7c1-f6be-32c966fd711c@twisted.org.uk> References: <b6fae531-edaf-d7c1-f6be-32c966fd711c@twisted.org.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000003764b805d8a0aed0 Content-Type: text/plain; charset="UTF-8" On Tue, Feb 22, 2022, 12:12 PM Pete French <pete@twisted.org.uk> wrote: > So, I have a number of machines in AWS. They are all of type r5a.xlarge > which is supposed to have 140 gig of instance syorage on it. All these > machines started life as clones of the same dirve, ro are runnign the > same OS kernel, and they all have mysql on them. Some of them also have > Apache and some other software, but I thought the config of the base OS > was the same. Certainly /boot/loader.conf and /etc/sysctl.conf are > indentical and its the same kernel and userland running on all of them. > > but on the mysql-only of the machines the instance drives have the worng > size, around a gig, and are not recognised properly. Example, heres iw > what it is supposed to look like (from one of the Apache machines) > > root@sydney01:/usr/home/webadmin # diskinfo -v nda2 > nda2 > 512 # sectorsize > 150000000000 # mediasize in bytes (140G) > 292968750 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > Amazon EC2 NVMe Instance Storage # Disk descr. > AWSB7ABDF8FE8D0597AF # Disk ident. > nvme2 # Attachment > Yes # TRIM/UNMAP support > 0 # Rotation rate in RPM > > and here is one of the ones which is wrong... > > root@serpentine-sydy:~ # diskinfo -v nda2 > nda2 > 512 # sectorsize > 886571008 # mediasize in bytes (846M) > 1731584 # mediasize in sectors > 131072 # stripesize > 0 # stripeoffset > No # TRIM/UNMAP support > Unknown # Rotation rate in RPM > What does 'nvmecontrol identify nvme2' and 'nvmecontrol identify nda2' say for each? Warner But both machines, in dmesg, have lines which look like this: > > nda2 at nvme2 bus 0 scbus2 target 0 lun 1 > nda2: <Amazon EC2 NVMe Instance Storage 0 AWSB7ABDF8FE8D0597AF> > nda2: Serial Number AWSB7ABDF8FE8D0597AF > nda2: nvme version 1.0 x0 (max x0) lanes PCIe Gen0 (max Gen0) link > nda2: 143051MB (292968750 512 byte sectors) > > So on both of then the detection says its the right size. > > The oddest this is that this is completely reproducible betwene data > centres - the above machines above are in Sydney, but I get precisely > the same result from the machines in North Virginia. So its something > about the config but what on earth could it be ? > > I am very puzzled - I have a set of database-nly machine sin Frankfurt, > and they behave fine! Poissibly I should just clone those to Australia > and the US, but I would like to find out what the magic difference is > between them. > > -pete. > > > --0000000000003764b805d8a0aed0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">On Tue, Feb 22, 2022, 12:12 PM Pete French <<a href= =3D"mailto:pete@twisted.org.uk">pete@twisted.org.uk</a>> wrote:<br></div= ><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1= px #ccc solid;padding-left:1ex">So, I have a number of machines in AWS. The= y are all of type r5a.xlarge <br> which is supposed to have 140 gig of instance syorage on it. All these <br> machines started life as clones of the same dirve, ro are runnign the <br> same OS kernel, and they all have mysql on them. Some of them also have <br= > Apache and some other software, but I thought the config of the base OS <br= > was the same. Certainly /boot/loader.conf and /etc/sysctl.conf are <br> indentical and its the same kernel and userland running on all of them.<br> <br> but on the mysql-only of the machines the instance drives have the worng <b= r> size, around a gig, and are not recognised properly. Example, heres iw <br> what it is supposed to look like (from one of the Apache machines)<br> <br> root@sydney01:/usr/home/webadmin # diskinfo -v nda2<br> nda2<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 512=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0# sectorsize<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 150000000000=C2=A0 =C2=A0 # mediasize in bytes = (140G)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 292968750=C2=A0 =C2=A0 =C2=A0 =C2=A0# mediasize= in sectors<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0# stripesize<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0# stripeoffset<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Amazon EC2 NVMe Instance Storage=C2=A0 =C2=A0 = =C2=A0 =C2=A0 # Disk descr.<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 AWSB7ABDF8FE8D0597AF=C2=A0 =C2=A0 # Disk ident.= <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 nvme2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0#= Attachment<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Yes=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0# TRIM/UNMAP support<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0# Rotation rate in RPM<br> <br> and here is one of the ones which is wrong...<br> <br> root@serpentine-sydy:~ # diskinfo -v nda2<br> nda2<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0512=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0# sectorsize<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0886571008=C2=A0 =C2=A0 =C2=A0 =C2=A0# med= iasize in bytes (846M)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01731584=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= # mediasize in sectors<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0131072=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = # stripesize<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0# stripeoffset<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0No=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 # TRIM/UNMAP support<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Unknown=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= # Rotation rate in RPM<br></blockquote></div></div><div dir=3D"auto"><br></= div><div dir=3D"auto">What does 'nvmecontrol identify nvme2' and &#= 39;nvmecontrol identify nda2' say for each?</div><div dir=3D"auto"><br>= </div><div dir=3D"auto">Warner</div><div dir=3D"auto"><br></div><div dir=3D= "auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style= =3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> But both machines, in dmesg, have lines which look like this:<br> <br> nda2 at nvme2 bus 0 scbus2 target 0 lun 1<br> nda2: <Amazon EC2 NVMe Instance Storage 0 AWSB7ABDF8FE8D0597AF><br> nda2: Serial Number AWSB7ABDF8FE8D0597AF<br> nda2: nvme version 1.0 x0 (max x0) lanes PCIe Gen0 (max Gen0) link<br> nda2: 143051MB (292968750 512 byte sectors)<br> <br> So on both of then the detection says its the right size.<br> <br> The oddest this is that this is completely reproducible betwene data <br> centres - the above machines above are in Sydney, but I get precisely <br> the same result from the machines in North Virginia. So its something <br> about the config but what on earth could it be ?<br> <br> I am very puzzled - I have a set of database-nly machine sin Frankfurt, <br= > and they behave fine! Poissibly I should just clone those to Australia <br> and the US, but I would like to find out what the magic difference is <br> between them.<br> <br> -pete.<br> <br> <br> </blockquote></div></div></div> --0000000000003764b805d8a0aed0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrye3mBj5Ej=vjoUUtKx%2BqNX%2BXBXq15j6B0FCnZDZw54Q>