Date: Fri, 8 Feb 2019 19:35:04 +0000 From: Eric Bautsch <eric.bautsch@pobox.com> To: freebsd-xen@freebsd.org Subject: Issues with XEN and ZFS Message-ID: <a3b338c7-d3ea-b552-2fbb-f3185f7dc96a@pobox.com>
next in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.
--------------ms090506020503050301080400
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Hi.
Brief abstract: I'm having ZFS/Xen interaction issues with the disks bein=
g=20
declared unusable by the dom0.
The longer bit:
I'm new to FreeBSD, so my apologies for all the stupid questions. I'm try=
ing to=20
migrate from Linux as my virtual platform host (very bad experiences with=
=20
stability, let's leave it at that). I'm hosting mostly Solaris VMs (that =
being=20
my choice of OS, but again, Betamax/VHS, need I say more), as well as a W=
indows=20
VM (because I have to) and a Linux VM (as a future desktop via thin clien=
ts as=20
and when I have to retire my SunRay solution which also runs on a VM for =
lack of=20
functionality).
So, I got xen working on FreeBSD now after my newbie mistake was pointed =
out to me.
However, I seem to be stuck again:
I have, in this initial test server, only two disks. They are SATA hangin=
g off=20
the on-board SATA controller. The system is one of those Shuttle XPC cube=
s, an=20
older one I had hanging around with 16GB memory and I think 4 cores.
I've given the dom0 2GB of memory and 2 core to start with.
The root filesystem is zfs with a mirror between the two disks.
The entire thing is dead easy to blow away and re-install as I was very=20
impressed how easy the FreeBSD automatic installer was to understand and =
pick=20
up, so I have it all scripted. If I need to blow stuff away to test, no p=
roblem=20
and I can always get back to a known configuration.
As I only have two disks, I have created a zfs volume for the Xen domU th=
us:
zfs create -V40G -o volmode=3Ddev zroot/nereid0
The domU nereid is defined thus:
cat - << EOI > /export/vm/nereid.cfg
builder =3D "hvm"
name =3D "nereid"
memory =3D 2048
vcpus =3D 1
vif =3D [ 'mac=3D00:16:3E:11:11:51,bridge=3Dbridge0',
'mac=3D00:16:3E:11:11:52,bridge=3Dbridge1',
'mac=3D00:16:3E:11:11:53,bridge=3Dbridge2' ]
disk =3D [ '/dev/zvol/zroot/nereid0,raw,hda,rw' ]
vnc =3D 1
vnclisten =3D "0.0.0.0"
serial =3D "pty"
EOI
nereid itself also auto-installs, it's a Solaris 11.3 instance.
As it tries to install, I get this in the dom0:
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada1:ahcich1:0:0:0):=20
WRITE_FPDMA_QUEUED. ACB: 61 18 a0 ef 88 40 46 00 00 00 00 00
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada1:ahcich1:0:0:0): CAM s=
tatus:=20
CCB request was invalid
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada1:ahcich1:0:0:0): Error=
22,=20
Unretryable error
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada0:ahcich0:0:0:0):=20
WRITE_FPDMA_QUEUED. ACB: 61 10 58 9d 86 40 3b 00 00 00 00 00
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada0:ahcich0:0:0:0): CAM s=
tatus:=20
CCB request was invalid
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada0:ahcich0:0:0:0): Error=
22,=20
Unretryable error
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk devd[555]: Processing event '!syste=
m=3DZFS=20
subsystem=3DZFS type=3Dereport.fs.zfs.io class=3Dereport.fs.zfs.io=20
ena=3D8237899368669119489 pool=3Dzroot pool_guid=3D18382324475986588472 p=
ool_context=3D0=20
pool_failmode=3Dwait vdev_guid=3D18239716500353510442 vdev_type=3Ddisk=20
vdev_path=3D/dev/ada1p3 parent_guid=3D246995617967232936 parent_type=3Dmi=
rror=20
zio_err=3D22 zio_offset=3D509174525952 zio_size=3D8192 zio_objset=3D236 z=
io_object=3D1=20
zio_level=3D0 zio_blkid=3D1235478'
Feb 8 18:57:16 bianca.swangage.co.uk syslogd: exiting on signal 15
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada0:ahcich0:0:0:0):=20
WRITE_FPDMA_QUEUED. ACB: 61 10 58 9d 86 40 3b 00 00 00 00 00
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada0:ahcich0:0:0:0): CAM s=
tatus:=20
CCB request was invalid
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada0:ahcich0:0:0:0): Error=
22,=20
Unretryable error
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada1:ahcich1:0:0:0):=20
WRITE_FPDMA_QUEUED. ACB: 61 10 58 9d 86 40 3b 00 00 00 00 00
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada1:ahcich1:0:0:0): CAM s=
tatus:=20
CCB request was invalid
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
Feb 8 18:57:16 bianca.swangage.co.uk kernel: (ada1:ahcich1:0:0:0): Error=
22,=20
Unretryable error
Feb 8 18:57:16 bianca.swangage.co.uk last message repeated 4 times
I get thousands or tens of thousands of these.
That has now lead to the system becoming unusable.
I didn't manage to complete the installation this time, but I did manage =
once=20
for a different Solaris 11.3 domU (which also caused the same errors in t=
he dom0=20
during install but did complete) and once it runs, it still does happen, =
but not=20
quite as often.
The issue does appear to be disk activity, in this particular instance: w=
rites.
I'm wondering if I neglected a particular setting to make ZFS behave prop=
erly or=20
perhaps the VM?
Any pointers would be greatly appreciated.
Thanks.
Eric
--=20
=20
____
/ . Eric A. Bautsch
/-- __ ___ __________________________________=
____
/ / / / /
(_____/____(___(__________________/ email: eric.bautsch@pobox.co=
m
--------------ms090506020503050301080400
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC
DFIwggNiMIIC6KADAgECAggwtYJcB7vEnDAKBggqhkjOPQQDAzBSMQswCQYDVQQGEwJFUzEU
MBIGA1UECgwLU3RhcnRDb20gQ0ExLTArBgNVBAMMJFN0YXJ0Q29tIENlcnRpZmljYXRpb24g
QXV0aG9yaXR5IEVDQzAeFw0xNzA0MjgwODAwMzVaFw0zNzA0MjgwODAwMzVaMGkxCzAJBgNV
BAYTAkVTMRQwEgYDVQQKDAtTdGFydENvbSBDQTEpMCcGA1UECwwgU3RhcnRDb20gQ2VydGlm
aWNhdGlvbiBBdXRob3JpdHkxGTAXBgNVBAMMEFN0YXJ0Q29tIENDMiBJQ0EwdjAQBgcqhkjO
PQIBBgUrgQQAIgNiAAR7hlYvM7ymfqRetYHdncaz11zCyZQbJofX1jT1FiEsyKH7WFh7k9cN
BMbe9RUh7mq6EcCcP7rHdV1yhkx9CNT8KSSDHIIWB1RbmK5XtKvK4BLQ1pLUbzvGVz/YBYro
HK+jggFyMIIBbjBtBggrBgEFBQcBAQRhMF8wNQYIKwYBBQUHMAKGKWh0dHA6Ly9haWEuc3Rh
cnRjb21jYS5jb20vY2VydHMvY2FjYzIuY3J0MCYGCCsGAQUFBzABhhpodHRwOi8vb2NzcC5z
dGFydGNvbWNhLmNvbTAdBgNVHQ4EFgQUPLfG3okmWlcDidCvMGpGDgzq3GYwEgYDVR0TAQH/
BAgwBgEB/wIBADAfBgNVHSMEGDAWgBSeiMCybDMJy/8hfr/qnwiGu32qGTBBBgNVHSAEOjA4
MDYGBFUdIAAwLjAsBggrBgEFBQcCARYgaHR0cDovL3d3dy5zdGFydGNvbWNhLmNvbS9wb2xp
Y3kwNwYDVR0fBDAwLjAsoCqgKIYmaHR0cDovL2NybC5zdGFydGNvbWNhLmNvbS9zZnNjYWNj
Mi5jcmwwDgYDVR0PAQH/BAQDAgEGMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDAK
BggqhkjOPQQDAwNoADBlAjEAxVHjDb7E+HcRO7j3UZg3lyI/6MgNJuD/Fc/5HTtjZc5B0iVz
eeERiqV1sGJ/h9h8AjAlmjRwgkRXx8hJVcCzCCBl95zytvLdJdGPrBJHEaFJnsYX8FQZGB86
0clRb9QPXXQwggQzMIIDuKADAgECAghYh6dhuIrClTAKBggqhkjOPQQDAzBpMQswCQYDVQQG
EwJFUzEUMBIGA1UECgwLU3RhcnRDb20gQ0ExKTAnBgNVBAsMIFN0YXJ0Q29tIENlcnRpZmlj
YXRpb24gQXV0aG9yaXR5MRkwFwYDVQQDDBBTdGFydENvbSBDQzIgSUNBMB4XDTE3MDcwMzEw
MzcyOVoXDTE5MDcwMzAyNDcwMFowSDElMCMGCSqGSIb3DQEJARYWZXJpYy5iYXV0c2NoQHBv
Ym94LmNvbTEfMB0GA1UEAwwWZXJpYy5iYXV0c2NoQHBvYm94LmNvbTCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBAMghw1tHL4eLIUgcw9dnIy+JADzgH7YBJYhZWNH8J6Vq2kiU
UTpBjjTALMNWxj7PakNtMXjWHqnQjWESwwzlvnaQnvf2bBjYafiC8+D3oocW3OyaDOLVRDqx
dI2n+Zr2RNTZw2erl2/cHrToVvOkuqZVftcL8EocMHeLuaEUfgaXQBmFhUJpzDvPsSLp99fg
z5zOY+j3sHa6HOGke8NvR4bi8pKnkgCu5lo9HWHgVJ/Ip8Cqk2EzwaZ0DSGfpvfXtv+OuBqO
s6VBJ19TibT9wfFeYeoesgKnS73zQKLoZG3yKcfYfZs9TxS5BEhWDWr6JFP8hUlhL+ZUi+X9
AFNAAx8CAwEAAaOCAZ4wggGaMHQGCCsGAQUFBwEBBGgwZjA8BggrBgEFBQcwAoYwaHR0cDov
L2FpYS5zdGFydGNvbWNhLmNvbS9jZXJ0cy9zY2EuY2xpZW50MjIuY3J0MCYGCCsGAQUFBzAB
hhpodHRwOi8vb2NzcC5zdGFydGNvbWNhLmNvbTAdBgNVHQ4EFgQUS/x/U30ucvaPvk4aAXYu
Q8qcFskwCQYDVR0TBAIwADAfBgNVHSMEGDAWgBQ8t8beiSZaVwOJ0K8wakYODOrcZjBIBgNV
HSAEQTA/MD0GCysGAQQBgbU3AgIBMC4wLAYIKwYBBQUHAgEWIGh0dHA6Ly93d3cuc3RhcnRj
b21jYS5jb20vcG9saWN5MDsGA1UdHwQ0MDIwMKAuoCyGKmh0dHA6Ly9jcmwuc3RhcnRjb21j
YS5jb20vc2NhLWNsaWVudDIyLmNybDAOBgNVHQ8BAf8EBAMCBLAwHQYDVR0lBBYwFAYIKwYB
BQUHAwIGCCsGAQUFBwMEMCEGA1UdEQQaMBiBFmVyaWMuYmF1dHNjaEBwb2JveC5jb20wCgYI
KoZIzj0EAwMDaQAwZgIxAKbrgOkZ5i8pHnjkbxiyZbOvisCA9Z+0/DZjPybtrKlk3l/dl7dd
AqPaZHKFNjGkGgIxAITRkSRMx0zlIb1ajYqEe3lVeouUc253pu+FOlAr5qvvJjZ+Gyc4/7ud
YIdBYQb4KzCCBLEwggKZoAMCAQICEEzFbU1ZMWzGD67YyYtePFkwDQYJKoZIhvcNAQEMBQAw
fTELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3Vy
ZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxKTAnBgNVBAMTIFN0YXJ0Q29tIENlcnRp
ZmljYXRpb24gQXV0aG9yaXR5MB4XDTE3MDQxMTA3MzAwMVoXDTIyMDQxMTA3MzAwMVowUjEL
MAkGA1UEBhMCRVMxFDASBgNVBAoMC1N0YXJ0Q29tIENBMS0wKwYDVQQDDCRTdGFydENvbSBD
ZXJ0aWZpY2F0aW9uIEF1dGhvcml0eSBFQ0MwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAARtU/dM
PcdNPiBafCl30C/R50Onc2/k4XgCYiJJrpw3hTs0B0P/+SZAknB0QU2BcIee3+22c5Ju/2UC
meZoVbtekPtlX1g6CoN0fQbqaMUhBUIyxEFavSPBd3IQ4/Hi7KujggEEMIIBADAOBgNVHQ8B
Af8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB/wIBAjAyBgNVHR8EKzApMCegJaAjhiFodHRwOi8v
Y3JsLnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwZgYIKwYBBQUHAQEEWjBYMCQGCCsGAQUFBzAB
hhhodHRwOi8vb2NzcC5zdGFydHNzbC5jb20wMAYIKwYBBQUHMAKGJGh0dHA6Ly9haWEuc3Rh
cnRzc2wuY29tL2NlcnRzL2NhLmNydDAdBgNVHQ4EFgQUnojAsmwzCcv/IX6/6p8Ihrt9qhkw
HwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwDQYJKoZIhvcNAQEMBQADggIBAJmQ
Ot9REj3wllH11HazsdBP1WD+Wt98G651eGEhVIHupeiO3+fM9JUxYNXrqFCcjlaWABGFAOXv
BAeBta7vT05Hk8TIC9/r2StixF+aQvbWq0DGiPOfX29FWKJEe/e0QHrqebP29cwwh2WbzsPz
EBklt0xObEt+/6R7QB7kJkqYeggmFSOf7G6VBlKDMt1dabf2uWs0cYoNceBHWX+Bepkl+V16
MJ4eYtOOGMzfaqbszR7SP4zJWXVOXbSa0gvk8zcAveJBw2CQK8My7mKCPx3IEhtn70b5oEBU
kGZpEbSCEI9XCs3VHpnZuB3s7PimeULw0Vkvkug0GdWBvGPgyCwDCdp9oXonZmCDkKVn2IuI
PWpYfIwAD8hK6nn14mrHXnOcyr0BB93hC1Cv394cjnunn9DSEgTxLgQITvCcbVlZdUIBObMi
hFFCz2EkySSYp5HmpUTFG+LhY9B2TevagHztoDbOg7BBvbLzetZh7kM7Dxqp2EVDJ7qhU0So
OLH7zeMf93L8wmg9pK5niFIFY9KbRoc0s/lTl+o+pjSsIrXEOovlDBx3Eg4ysyyvQQErbuRD
0sRrpHYbaJySScoFK8nAEgkDO+Q4EC2/mlowjy5fM6X1+FmZFedPbd4G59OadRVe7sGEK/4G
nvu0BZUbbkUhfoFg/iAUGX83w93l5bRJMYIDizCCA4cCAQEwdTBpMQswCQYDVQQGEwJFUzEU
MBIGA1UECgwLU3RhcnRDb20gQ0ExKTAnBgNVBAsMIFN0YXJ0Q29tIENlcnRpZmljYXRpb24g
QXV0aG9yaXR5MRkwFwYDVQQDDBBTdGFydENvbSBDQzIgSUNBAghYh6dhuIrClTANBglghkgB
ZQMEAgEFAKCCAecwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcN
MTkwMjA4MTkzNTA0WjAvBgkqhkiG9w0BCQQxIgQggknWKDFlhGILIlPofz4aQlf6neemudui
74mxTZE0Xd8wbAYJKoZIhvcNAQkPMV8wXTALBglghkgBZQMEASowCwYJYIZIAWUDBAECMAoG
CCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggq
hkiG9w0DAgIBKDCBhAYJKwYBBAGCNxAEMXcwdTBpMQswCQYDVQQGEwJFUzEUMBIGA1UECgwL
U3RhcnRDb20gQ0ExKTAnBgNVBAsMIFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5
MRkwFwYDVQQDDBBTdGFydENvbSBDQzIgSUNBAghYh6dhuIrClTCBhgYLKoZIhvcNAQkQAgsx
d6B1MGkxCzAJBgNVBAYTAkVTMRQwEgYDVQQKDAtTdGFydENvbSBDQTEpMCcGA1UECwwgU3Rh
cnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxGTAXBgNVBAMMEFN0YXJ0Q29tIENDMiBJ
Q0ECCFiHp2G4isKVMA0GCSqGSIb3DQEBAQUABIIBAKMn7r8j7vKRqGV/1n2ZpIlK0o9HWV0z
zFgu/qdlUk4MLx2hk+UFHXHVWXDz+VBZUrx94sPhLEbwr82tZg2CMeLiFYY98tWW/2NlXiaR
ggOzYFMmATZm6zkaEqbprBM7ZPivDL6NKAIIctmFB6kfVJlHHtIhrNmWR31GDQFV6Nxo53w5
5dvXRDykMAlN+OvpzaVbezSfBDxcxRK7z2BS9z0bBdLDOaPJw6G9Z0FHulPnbjRRZApEw1xZ
KClCaXWi71Mu/c6b3c0hY1XCBhEiHQ3D7mRoltnZkPQjkSaglwgIvoJDxFSz2I9EvY3HPuLq
2hL7z5fprsSuMQUKlUQewqQAAAAAAAA=
--------------ms090506020503050301080400--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a3b338c7-d3ea-b552-2fbb-f3185f7dc96a>