From nobody Sun Sep 8 12:10:29 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X1phK1jl7z5Tfqk for ; Sun, 08 Sep 2024 12:10:41 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X1phK1QYFz5159; Sun, 8 Sep 2024 12:10:41 +0000 (UTC) (envelope-from theraven@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725797441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vUcwaxhOBJsMr40uB+Jw+QVkTlaAddWL7WcxiS4vrqM=; b=ouhusGIq3c06PrGkBFQ8JStir9NHwjzHwvRYoiLunwCzckBgw/yJ51Kmi7y5mWPrcchlR3 C6ZCIOds5LCmX81QnqXtXm0DS09VJSZc6iw3/pJiOSReClqIczvsBk4Elji89f3eP1Q4ji 5l5FxJKusHDLwRBHXatpBgaNGtByGA8NaYE3cOMkQnJ8+0wKEBQvMeHhl7nz8ia2PJQ7Ap 2vCg8SyPL4bN3Aw//liVTUo9rM3Rd1MTt5sPrZ9+uTntaYi72Hloe8ioReD0D0M5TANGcr HYwzubLrVpWOlYcLpp6RbgPd7X3mhUsEBARLxmdtFnIEAK5bSAKAoq8wh6LbGA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725797441; a=rsa-sha256; cv=none; b=JdkjhmORgfThahsty3Akl39EGJvD1XY2mlMcyaLxZ8NcYYg3u89hAXEu1RpUPFBGPsDZAt 8BxcibV+RCxqKMJmIo7TAwSvvqjd+FXtGrm0pQLcMFMyxx04RWkCCc4c1V43MzKZOaHzd/ jg1dQiP8av+KcaWPtsu3BuF0kUWgZ3C4otOGlzdW/5KtgDioVTRhImXgmDXgSjcXxP1XFq Jw+N8ZoRnqfo4qyyNIG1QhTsLRCjzMWbBCh3WDFxBZqkAS5ejTPTD0Jo7IWyJO13/ZO67i jYfk4xbPbL1BnAoi4GWAoPtG6FtcH8lRt2lZAvzU6AvD500hqmr3LcJAYvJDug== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725797441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vUcwaxhOBJsMr40uB+Jw+QVkTlaAddWL7WcxiS4vrqM=; b=e2WJpik5e7jwAAdq9nnjjvHsendOuImB72hpcAkc58DynVMfpGBTZTLoP0H1tXE7QxPgH4 xpYNwtDPFmbgrSlc3jsTK5LC5xhI/nkZMSpmRW7DbHUcUXJEpAz1Rc7k3JXJZ4AMnCwuC6 A2hzD+bQbkNeMuEGVMffkFP1ydvfs466rLJrC7H8zDhi9D8gMAdnrchrrSuZUdct3+0Qj6 QqZEXV4Uz4Kg3LRBD5TzhQ5+csPytGNEk/09av6Pqc5ayrlEq/YSN2q6cx9/ryIvv7kBt2 7/4whtcz8m3ayfCYj9S9FO53r/bDmAVl0PxYGxe39IsAtOA6iJJ/HHUuxpyDag== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4X1phK0r3SzPZx; Sun, 8 Sep 2024 12:10:41 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtpclient.apple (host109-155-136-107.range109-155.btcentralplus.com [109.155.136.107]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 68AC2655E; Sun, 08 Sep 2024 13:10:40 +0100 (BST) From: David Chisnall Message-Id: <1CF8CD26-51D8-4599-B398-DC333D3A9D1C@FreeBSD.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_77A5D516-896F-451A-85CC-D471D0F0682F" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: FreeBSD+samba as a time machine server for OSX/Sonoma? Date: Sun, 8 Sep 2024 13:10:29 +0100 In-Reply-To: <9F9D21A4-5747-4F2A-960E-2CF826C8BEC4@FreeBSD.org> Cc: freebsd-hackers@freebsd.org To: Craig Leres References: <8E0CDC45-6521-4973-A349-9B5824C75863@freebsd.org> <9F9D21A4-5747-4F2A-960E-2CF826C8BEC4@FreeBSD.org> X-Mailer: Apple Mail (2.3776.700.51) --Apple-Mail=_77A5D516-896F-451A-85CC-D471D0F0682F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I have opened https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281360 = to track this. Reverting to samba416, I can back up in a new share, but I cannot back = up to the share that had my existing backups in it. This may be a = permissions issue or some problem withe metadata getting out of sync: = moving the backupbundle from the old share to the new one does not fix = it. Once I have a new complete backup, I will try comparing the = metadata and seeing what the differences are. So far, the main difference appears to be that the new backup creates a = disk image bundle with 500 MiB bands whereas the old one was using 8 MiB = bands. This change is presumably because APFS, unlike HFS+, supports = holes in files, but I wonder if there=E2=80=99s a problem with Samba and = very large directories? There were 47,486 bands in the old disk image. David > On 8 Sep 2024, at 12:48, David Chisnall wrote: >=20 > A little bit more debugging, and a working hypothesis: >=20 > I have tried creating a new Time Machine share (empty directory) and = pointing the Mac at it. On the first backup, it creates a = `.com.apple.timemachine.supported-{uuid}` file and a = `{uuid}-{date}.sparsebundle` directory. It appears to create a valid = sparse bundle, but other posts suggest that it then renames this to = `{computername}.backupbundle`. There is a Samba setting: = `fruit:posix_rename` that is supposed to control this. It appears to = fail at the point where it should do this rename. =20 >=20 > If I mount the same share, I can reproduce this: >=20 > $ mkdir tmp > $ touch tmp/foo > $ mv tmp fmp > mv: rename tmp to fmp: Operation not supported >=20 > So it appears that something in the FreeBSD port of Samba has broken = the ability to rename directories. =20 >=20 > This appears to be recent breakage. Reverying to net/samba416 fixes = this bit, at least, and I can now back up to a pristine share. >=20 > David >=20 >> On 7 Sep 2024, at 08:28, David Chisnall wrote: >>=20 >> I believe this was broken by a macOS update around February. I=E2=80=99= ve been trying to debug for a while. I=E2=80=99ve opened an Apple issue = (FB14500950, for any Apple folks) but it shows up as few people = reporting it. I posted on Mastodon and several people reported that Time = Machine is broken and recommended Carbon Copy Cloner as an alternative. = I would like to see it fixed, but it probably needs some more debugging = by Apple folks.=20 >>=20 >> It stopped working for me with no changes on the server and I can = reproduce the failures on two different Macs. >>=20 >> Things I have tried: >>=20 >> - Upgrading Samba from 4.16 to 4.19 >> - Upgrading FreeBSD from 13.x to 14.1 >> - Setting the SMB timeout sysctls to larger values on macOS. >> - Turning up the SMB debug sysctls on macOS to see if there=E2=80=99s = more info >> - Turning up the Samba logging level. >> - Verifying the backups >> - Watching smbinfo the server. >> - Updating macOS to the latest version >> - Connecting to the server with Finder and checking I can access = files on the shares and that they have the right permissions. >>=20 >> Samba doesn=E2=80=99t report any errors (I don=E2=80=99t know if = there=E2=80=99s a way to force Samba to report permission-denied = things). >>=20 >> It appears that the Mac acquires a load of read-only locks and so = does a lot of reads, but for some reason it appears to fail the first = write. Even with a verify, it looks like it completes the verification = bit but then fails to write to the plist file.=20 >>=20 >> With the increased debugging, I see this in the macOS Comsole: >>=20 >> default 14:12:26.297714+0100 kernel smb2fs_smb_cmpd_create: = smb2fs_smb_ntcreatex failed 13 >> default 14:12:26.301301+0100 kernel smb2fs_smb_cmpd_create: = smb2fs_smb_ntcreatex failed 13 >> default 14:12:26.310563+0100 kernel smb2fs_smb_cmpd_query: = smb2_smb_query_info (single request) failed 45 >> default 14:12:26.318319+0100 kernel smb2fs_smb_cmpd_query: = smb2_smb_query_info (single request) failed 45 >> default 14:12:26.326850+0100 backupd -[DIStatFS = initWithFileDescriptor:error:]: File system is smbfs >> default 14:12:26.542645+0100 kernel smbfs_vnop_access: 501 = not authorized to access TheRooT : action =3D 0x80 >> default 14:12:26.542682+0100 kernel smbfs_vnop_access: = TheRooT action =3D 0x80 denied >> default 14:12:26.543622+0100 kernel smbfs_vnop_access: 501 = not authorized to access TheRooT : action =3D 0x80 >> default 14:12:26.543657+0100 kernel smbfs_vnop_access: = TheRooT action =3D 0x80 denied >> default 14:12:26.543690+0100 kernel smbfs_vnop_access: 501 = not authorized to access TheRooT : action =3D 0x80 >> default 14:12:26.543697+0100 kernel smbfs_vnop_access: = TheRooT action =3D 0x80 denied >> default 14:12:26.543725+0100 kernel smbfs_vnop_access: 501 = not authorized to access TheRooT : action =3D 0x80 >> default 14:12:26.543730+0100 kernel smbfs_vnop_access: = TheRooT action =3D 0x80 denied >> default 14:12:26.544085+0100 kernel smbfs_vnop_access: 501 = not authorized to access TheRooT : action =3D 0x80 >>=20 >> So it looks as if it is a permission issue. Maybe the mcOS SMB client = has started using some bit of the protocol that Samba on FreeBSD = doesn=E2=80=99t support for ACLs? >>=20 >> David >>=20 >>> On 6 Sep 2024, at 22:48, Craig Leres wrote: >>>=20 >>> =EF=BB=BFLast year you guys helped me get this to work with = samba416. I recently tried to upgrade to samba419 and so far I'm = unsuccessful. The error is "The backup disk image could not be created" = and I'm running 14.1. >>>=20 >>> I'm using the same port build options with 4.16 and 4.19: >>>=20 >>> FAM >>> PYTHON3 >>> QUOTAS >>> SYSLOG >>> UTMP >>> GSSAPI_BUILTIN >>> AVAHI >>> FRUIT >>>=20 >>> Having learned my lesson when I upgraded from 4.13 to 4.16, I = removed the old backups from the zfs volume on the server before = starting. I've also learned the rule that you need to delete and = reattach the share on the mac side when you change the samba config. >>>=20 >>> Appended is the config that works with 4.16 (but not 4.19) >>>=20 >>> Craig >>>=20 >>> [global] >>> workgroup =3D XYZ >>> security =3D user >>> netbios name =3D red >>> server string =3D red.example.net >>> hostname lookups =3D no >>> server role =3D standalone server >>>=20 >>> interfaces =3D ixl0 lo0 >>> bind interfaces only =3D yes >>>=20 >>> load printers =3D no >>> show add printer wizard =3D no >>> time server =3D yes >>> use mmap =3D yes >>>=20 >>> dos charset =3D 850 >>> unix charset =3D UTF-8 >>> mangled names =3D no >>>=20 >>> #log level =3D 3 >>> #log file =3D /tmp/samba.log >>> vfs objects =3D catia fruit streams_xattr zfsacl >>>=20 >>> fruit:model =3D MacSamba >>> fruit:resource =3D file >>> fruit:metadata =3D netatalk >>> fruit:nfs_aces =3D yes >>> fruit:copyfile =3D no >>> fruit:aapl =3D yes >>> fruit:zero_file_id =3D yes >>>=20 >>> inherit permissions =3D yes >>>=20 >>>=20 >>> [Time Machine] >>> path =3D /backups/mini >>> read only =3D no >>> guest ok =3D no >>> writeable =3D yes >>> browseable =3D yes >>> fruit:resource =3D file >>> fruit:time machine =3D yes >>> valid users =3D backup-mini >>> max disk size 512G >>>=20 >>> hosts allow =3D 10.0.0.19 >>>=20 >=20 --Apple-Mail=_77A5D516-896F-451A-85CC-D471D0F0682F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 I have = opened https:= //bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281360 to track = this.

Reverting to samba416, I can back up in a new = share, but I cannot back up to the share that had my existing backups in = it.  This may be a permissions issue or some problem withe metadata = getting out of sync: moving the backupbundle from the old share to the = new one does not fix it.  Once I have a new complete backup, I will = try comparing the metadata and seeing what the differences = are.

So far, the main difference appears to be = that the new backup creates a disk image bundle with 500 MiB bands = whereas the old one was using 8 MiB bands.  This change is = presumably because APFS, unlike HFS+, supports holes in files, but I = wonder if there=E2=80=99s a problem with Samba and very large = directories?  There were 47,486 bands in the old disk = image.

David

On 8 Sep 2024, at 12:48, David Chisnall = <theraven@FreeBSD.org> wrote:

A = little bit more debugging, and a working = hypothesis:

I have tried creating a new Time Machine = share (empty directory) and pointing the Mac at it.  On the first = backup, it creates a `.com.apple.timemachine.supported-{uuid}` file and = a `{uuid}-{date}.sparsebundle` directory.  It appears to create a = valid sparse bundle, but other posts suggest that it then renames this = to `{computername}.backupbundle`.  There is a Samba setting: = `fruit:posix_rename` that is supposed to control this.  It appears = to fail at the point where it should do this rename. =  

If I mount the same share, I can = reproduce this:

$ mkdir tmp
$ = touch tmp/foo
$ mv tmp fmp
mv: rename tmp to fmp: = Operation not supported

So it appears = that something in the FreeBSD port of Samba has broken the ability to = rename directories.  

This appears to be = recent breakage.  Reverying to net/samba416 fixes this bit, at = least, and I can now back up to a pristine = share.

David

On 7 Sep 2024, at 08:28, David Chisnall = <theraven@FreeBSD.org> wrote:

I believe this was broken by a macOS = update around February. I=E2=80=99ve been trying to debug for a while. = I=E2=80=99ve opened an Apple issue (FB14500950, for any Apple folks) but = it shows up as few people reporting it. I posted on Mastodon and several = people reported that Time Machine is broken and recommended Carbon Copy = Cloner as an alternative. I would like to see it fixed, but it probably = needs some more debugging by Apple folks. 

It stopped working for me with no = changes on the server and I can reproduce the failures on two different = Macs.

Things I have = tried:

 - = Upgrading Samba from 4.16 to 4.19
 - = Upgrading FreeBSD from 13.x to 14.1
 - = Setting the SMB timeout sysctls to larger values on macOS.
 - Turning up the SMB debug sysctls on macOS to see if = there=E2=80=99s more info
 - Turning up the = Samba logging level.
 - Verifying the = backups
 - Watching smbinfo the = server.
 - Updating macOS to the latest = version
 - Connecting to the server with = Finder and checking I can access files on the shares and that they have = the right permissions.

Samba doesn=E2=80=99t report any errors (I don=E2=80=99t = know if there=E2=80=99s a way to force Samba to report permission-denied = things).

It appears = that the Mac acquires a load of read-only locks and so does a lot of = reads, but for some reason it appears to fail the first write. Even with = a verify, it looks like it completes the verification bit but then fails = to write to the plist file. 

With the increased debugging, I see this in the macOS = Comsole:

default 14:12:26.297714+0100 = kernel smb2fs_smb_cmpd_create: smb2fs_smb_ntcreatex failed 13 default 14:12:26.301301+0100 kernel smb2fs_smb_cmpd_create: = smb2fs_smb_ntcreatex failed 13 default 14:12:26.310563+0100 kernel smb2fs_smb_cmpd_query: = smb2_smb_query_info (single request) failed 45 default 14:12:26.318319+0100 kernel smb2fs_smb_cmpd_query: = smb2_smb_query_info (single request) failed 45 default 14:12:26.326850+0100 backupd -[DIStatFS = initWithFileDescriptor:error:]: File system is smbfs default 14:12:26.542645+0100 kernel smbfs_vnop_access: 501 not = authorized to access TheRooT : action =3D 0x80 default 14:12:26.542682+0100 kernel smbfs_vnop_access: TheRooT = action =3D 0x80 denied default 14:12:26.543622+0100 kernel smbfs_vnop_access: 501 not = authorized to access TheRooT : action =3D 0x80 default 14:12:26.543657+0100 kernel smbfs_vnop_access: TheRooT = action =3D 0x80 denied default 14:12:26.543690+0100 kernel smbfs_vnop_access: 501 not = authorized to access TheRooT : action =3D 0x80 default 14:12:26.543697+0100 kernel smbfs_vnop_access: TheRooT = action =3D 0x80 denied default 14:12:26.543725+0100 kernel smbfs_vnop_access: 501 not = authorized to access TheRooT : action =3D 0x80 default 14:12:26.543730+0100 kernel smbfs_vnop_access: TheRooT = action =3D 0x80 denied default 14:12:26.544085+0100 kernel smbfs_vnop_access: 501 not = authorized to access TheRooT : action =3D 0x80

So it looks as if it is a = permission issue. Maybe the mcOS SMB client has started using some bit = of the protocol that Samba on FreeBSD doesn=E2=80=99t support for = ACLs?

David

On 6 Sep 2024, at 22:48, Craig = Leres <leres@freebsd.org> = wrote:

=EF=BB=BFLast year you guys helped me get this to work = with samba416. I recently tried to upgrade to samba419 and so far I'm = unsuccessful. The error is "The backup disk image could not be created" = and I'm running 14.1.

I'm using the = same port build options with 4.16 and = 4.19:

=    FAM
=    PYTHON3
=    QUOTAS
=    SYSLOG
=    UTMP
=    GSSAPI_BUILTIN
=    AVAHI
=    FRUIT

Having learned = my lesson when I upgraded from 4.13 to 4.16, I removed the old backups = from the zfs volume on the server before starting. I've also learned the = rule that you need to delete and reattach the share on the mac side when = you change the samba config.

Appended = is the config that works with 4.16 (but not = 4.19)

      =  Craig

[global]
=    workgroup =3D XYZ
=    security =3D user
=    netbios name =3D red
=    server string =3D red.example.net
=    hostname lookups =3D no
=    server role =3D standalone = server

   interfaces =3D = ixl0 lo0
   bind interfaces only =3D = yes

   load printers =3D = no
   show add printer wizard =3D = no
   time server =3D = yes
   use mmap =3D = yes

   dos charset =3D = 850
   unix charset =3D = UTF-8
   mangled names =3D = no

   #log level =3D = 3
   #log file =3D = /tmp/samba.log
   vfs objects =3D catia = fruit streams_xattr zfsacl

=    fruit:model =3D MacSamba
=    fruit:resource =3D file
=    fruit:metadata =3D netatalk
=    fruit:nfs_aces =3D yes
=    fruit:copyfile =3D no
=    fruit:aapl =3D yes
=    fruit:zero_file_id =3D = yes

   inherit = permissions =3D = yes


[Time = Machine]
   path =3D = /backups/mini
   read only =3D = no
   guest ok =3D no
=    writeable =3D yes
=    browseable =3D yes
=    fruit:resource =3D file
=    fruit:time machine =3D yes
=    valid users =3D backup-mini
=    max disk size 512G

=    hosts allow =3D = 10.0.0.19



= --Apple-Mail=_77A5D516-896F-451A-85CC-D471D0F0682F--