Date: Tue, 21 Feb 2017 18:15:06 +0100 From: Dirk-Willem van Gulik <dirkx@webweaving.org> To: "Kenneth D. Merry" <ken@FreeBSD.ORG> Cc: freebsd-hackers@freebsd.org Subject: Re: sa and mpt in FreeBSD 11.0-RELEASE-p2 Message-ID: <9A1C3971-0614-4E7C-9A99-C40346D794CB@webweaving.org> In-Reply-To: <20170220205621.GA14597@mithlond.kdm.org> References: <C2E846C2-D96B-4429-8DC9-5612EF529F3E@webweaving.org> <20170220205621.GA14597@mithlond.kdm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 20 Feb 2017, at 21:56, Kenneth D. Merry <ken@FreeBSD.ORG> wrote:
>=20
> On Mon, Feb 20, 2017 at 21:04:11 +0100, Dirk-Willem van Gulik wrote:
>> A machine, recently upgraded from 8.4 to FreeBSD 11.0-RELEASE-p2 =
seems to have lost pretty much any and all performance on mpt(4) with =
its attached tape drives
>>=20
>> Read performance is around 50Mbyte/second - and write a paltry =
100-200kbyte/second (and occasionally hitting 800kbyte/second) (from/to =
memory disk&/dev/null; no risk of shoe shining, compression off).
>>=20
>> SCSI bus looks happy; with no kernel messages. Output of the DLT =
script below.
>>=20
>> Normal LTO drives in a tape robot, G9 server; pretty much all SAS =
*except* for a U320 to tape drive with the performance issue. Active =
terminator.
>>=20
>> Does this ring a bell with anyone? Was anything changed in either the =
sa or mpt driver since 8.x ?=20
>>=20
>> One odd thing is that a 'dd(1)' -without- a block size (compression =
is off, data is a prepared urandom file on md disk) writes much faster =
-- while on 8.x one -had- to use a sane blockwise to get the 'normal' =
speed of around 120Mbyte/second. Could it be that one needs to fiddle =
with MAXPHYS (which AFAIK is a readonly sysctl).
>>=20
>=20
> What blocksize are you using? What blocksize were you using before? =
What
> application do you normally use to talk to the tape drive?
Amanda (or plain DD); we increased Amanda=E2=80=99s default of 32 to 64k =
(which stopped any and all shoe shining at that time. It was tested in =
2013 (so likely in FreeBSD 8) up to 16 Mbyte in factors of 2 according =
to our notes - but with no further improvements in performance - and =
left at 64k).
The actual drive (mt status -v) reports (prior to increasing MAXPHYS) on =
11.0-p2 with default kernel config:
> Drive: sa0: <QUANTUM ULTRIUM 4 W52T> Serial Number: HU1027B53Y
> ---------------------------------
> Mode Density Blocksize bpi Compression
> Current: 0x46:LTO-4 variable 323215 enabled (0x1)
> ---------------------------------
> Current Driver State: at rest.
> ---------------------------------
> Partition: 0 Calc File Number: 0 Calc Record Number: 0
> Residual: 0 Reported File Number: 0 Reported Record Number: 0
> Flags: BOP
> ---------------------------------
> Tape I/O parameters:
> Maximum I/O size allowed by driver and controller (maxio): 131072 =
bytes
> Maximum I/O size reported by controller (cpi_maxio): 131072 bytes
> Maximum block size supported by tape drive and media (max_blk): =
16777215 bytes
> Minimum block size supported by tape drive and media (min_blk): 1 =
bytes
> Block granularity supported by tape drive and media (blk_gran): 0 =
bytes
> Maximum possible I/O size (max_effective_iosize): 131072 bytes
=E2=80=A6.
> If you want to re-enable I/O splitting (which I should have disabled =
by
> now), you can set the following loader tunable in /boot/loader.conf:
>=20
> kern.cam.sa.allow_io_split=3D1
Ok - this changes behaviour - non-blocksized limited writes are now up; =
2Mbyte/second; any block-sized write is 200kb/second (regardless of bs).
> If so, you probably want to look at the blocksize you're using, and =
bump up
> MAXPHYS in your kernel configuration to a larger value.
Post MAXPHYS & DFLTPHYS increase to 4MB (4x1024x1024) (as 16MB caused =
the kernel to hang during boot just post usb/knd - likely as it is there =
that it starts on SAS disk & tape discovery) I get
as =E2=80=98mt status -v=E2=80=99 the output:
> Tape I/O parameters:
> Maximum I/O size allowed by driver and controller (maxio): 1372160 =
bytes
> Maximum I/O size reported by controller (cpi_maxio): 1372160 bytes
> Maximum block size supported by tape drive and media (max_blk): =
16777215 bytes
> Minimum block size supported by tape drive and media (min_blk): 1 =
bytes
> Block granularity supported by tape drive and media (blk_gran): 0 =
bytes
> Maximum possible I/O size (max_effective_iosize): 1372160 bytes
And see the same speeds; non blocksize limited (with dd) are =
2.7Mbyte/second (so slightly higher); the blocksized writes are better =
(but still) low; 200kbyte/second for 16k, 600k for 32k, 800k for 64k, =
500k for 128k, 1M at 256k, 1M at 1024k. The writes at 2048k and 4096k =
fail with a =E2=80=98hardware failure asc:44,0 =E2=80=94 tape is now =
frozen=E2=80=9D error).
This is with `kern.cam.sa.allow_io_split=3D1=E2=80=99 in =
/boot/loader.conf=E2=80=99. Doing so without that (after a reboot) give =
substantially similar results: (1.8Mbyte/second (0), 450k (16k), 560k =
(32k), 0.9M (64), 560k (128k), 730k (256k) and 600k (1M) =E2=80=94 it =
fairly errors out with a si_iosize_max=3D1372160 for sizes above that.
Dw
#!/bin/sh
set -x
# create a ram disk of about 1 Gbyte with a noncompressible file on it.
#
mdconfig -a -s 1200M
newfs md0
mount /dev/md0 /mnt
dd if=3D/dev/urandom bs=3D1m count=3D1024 of=3D/mnt/test-1024m
for i in "" bs=3D16k bs=3D32k bs=3D64k bs=3D128k bs=3D256k bs=3D1024k =
bs=3D2048k bs=3D4196k
do
mt rewind
echo Run started with ${i:-no blocksize set}
dd if=3D/mnt/test-1024m $i of=3D/dev/nsa0
done
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9A1C3971-0614-4E7C-9A99-C40346D794CB>