Date: Sun, 8 Sep 2024 14:29:04 +0200 From: Dimitry Andric <dimitry@andric.com> To: David Chisnall <theraven@FreeBSD.org> Cc: Craig Leres <leres@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: FreeBSD+samba as a time machine server for OSX/Sonoma? Message-ID: <63540459-7D2E-4B70-962D-957550D294A3@andric.com> In-Reply-To: <1CF8CD26-51D8-4599-B398-DC333D3A9D1C@FreeBSD.org> References: <c7183af3-4a8b-4f12-848f-09f11e8b0e8f@freebsd.org> <8E0CDC45-6521-4973-A349-9B5824C75863@freebsd.org> <9F9D21A4-5747-4F2A-960E-2CF826C8BEC4@FreeBSD.org> <1CF8CD26-51D8-4599-B398-DC333D3A9D1C@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] I have never needed to explicitly set fruit:posix_rename, my Time Machine backups work completely fine without it (maybe it defaults to on now?). I have been using Samba 4.19 since the port came out too. The only problem I encountered was with samba416 and fruit:zero_file_id, which severely borked TM shares, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269883 for the gory details. Since I don't have access to a Sonoma Mac on my network, I have used Windows instead to make a bunch of subdirectories in a Samba share with fruit:time machine enabled, but I can rename them at any level without issue. -Dimitry > On 8 Sep 2024, at 14:10, David Chisnall <theraven@FreeBSD.org> wrote: > > I have opened https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=281360 to track this. > > Reverting to samba416, I can back up in a new share, but I cannot back up to the share that had my existing backups in it. This may be a permissions issue or some problem withe metadata getting out of sync: moving the backupbundle from the old share to the new one does not fix it. Once I have a new complete backup, I will try comparing the metadata and seeing what the differences are. > > So far, the main difference appears to be that the new backup creates a disk image bundle with 500 MiB bands whereas the old one was using 8 MiB bands. This change is presumably because APFS, unlike HFS+, supports holes in files, but I wonder if there’s a problem with Samba and very large directories? There were 47,486 bands in the old disk image. > > David > >> On 8 Sep 2024, at 12:48, David Chisnall <theraven@FreeBSD.org> wrote: >> >> A little bit more debugging, and a working hypothesis: >> >> I have tried creating a new Time Machine share (empty directory) and pointing the Mac at it. On the first backup, it creates a `.com.apple.timemachine.supported-{uuid}` file and a `{uuid}-{date}.sparsebundle` directory. It appears to create a valid sparse bundle, but other posts suggest that it then renames this to `{computername}.backupbundle`. There is a Samba setting: `fruit:posix_rename` that is supposed to control this. It appears to fail at the point where it should do this rename. >> >> If I mount the same share, I can reproduce this: >> >> $ mkdir tmp >> $ touch tmp/foo >> $ mv tmp fmp >> mv: rename tmp to fmp: Operation not supported >> >> So it appears that something in the FreeBSD port of Samba has broken the ability to rename directories. >> >> This appears to be recent breakage. Reverying to net/samba416 fixes this bit, at least, and I can now back up to a pristine share. >> >> David >> >>> On 7 Sep 2024, at 08:28, David Chisnall <theraven@FreeBSD.org> wrote: >>> >>> I believe this was broken by a macOS update around February. I’ve been trying to debug for a while. I’ve opened an Apple issue (FB14500950, for any Apple folks) but it shows up as few people reporting it. I posted on Mastodon and several people reported that Time Machine is broken and recommended Carbon Copy Cloner as an alternative. I would like to see it fixed, but it probably needs some more debugging by Apple folks. >>> >>> It stopped working for me with no changes on the server and I can reproduce the failures on two different Macs. >>> >>> Things I have tried: >>> >>> - Upgrading Samba from 4.16 to 4.19 >>> - Upgrading FreeBSD from 13.x to 14.1 >>> - Setting the SMB timeout sysctls to larger values on macOS. >>> - Turning up the SMB debug sysctls on macOS to see if there’s more info >>> - Turning up the Samba logging level. >>> - Verifying the backups >>> - Watching smbinfo the server. >>> - Updating macOS to the latest version >>> - Connecting to the server with Finder and checking I can access files on the shares and that they have the right permissions. >>> >>> Samba doesn’t report any errors (I don’t know if there’s a way to force Samba to report permission-denied things). >>> >>> It appears that the Mac acquires a load of read-only locks and so does a lot of reads, but for some reason it appears to fail the first write. Even with a verify, it looks like it completes the verification bit but then fails to write to the plist file. >>> >>> With the increased debugging, I see this in the macOS Comsole: >>> >>> default 14:12:26.297714+0100 kernel smb2fs_smb_cmpd_create: smb2fs_smb_ntcreatex failed 13 >>> default 14:12:26.301301+0100 kernel smb2fs_smb_cmpd_create: smb2fs_smb_ntcreatex failed 13 >>> default 14:12:26.310563+0100 kernel smb2fs_smb_cmpd_query: smb2_smb_query_info (single request) failed 45 >>> default 14:12:26.318319+0100 kernel smb2fs_smb_cmpd_query: smb2_smb_query_info (single request) failed 45 >>> default 14:12:26.326850+0100 backupd -[DIStatFS initWithFileDescriptor:error:]: File system is smbfs >>> default 14:12:26.542645+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 >>> default 14:12:26.542682+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied >>> default 14:12:26.543622+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 >>> default 14:12:26.543657+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied >>> default 14:12:26.543690+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 >>> default 14:12:26.543697+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied >>> default 14:12:26.543725+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 >>> default 14:12:26.543730+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied >>> default 14:12:26.544085+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 >>> >>> So it looks as if it is a permission issue. Maybe the mcOS SMB client has started using some bit of the protocol that Samba on FreeBSD doesn’t support for ACLs? >>> >>> David >>> >>>> On 6 Sep 2024, at 22:48, Craig Leres <leres@freebsd.org> wrote: >>>> >>>> Last year you guys helped me get this to work with samba416. I recently tried to upgrade to samba419 and so far I'm unsuccessful. The error is "The backup disk image could not be created" and I'm running 14.1. >>>> >>>> I'm using the same port build options with 4.16 and 4.19: >>>> >>>> FAM >>>> PYTHON3 >>>> QUOTAS >>>> SYSLOG >>>> UTMP >>>> GSSAPI_BUILTIN >>>> AVAHI >>>> FRUIT >>>> >>>> Having learned my lesson when I upgraded from 4.13 to 4.16, I removed the old backups from the zfs volume on the server before starting. I've also learned the rule that you need to delete and reattach the share on the mac side when you change the samba config. >>>> >>>> Appended is the config that works with 4.16 (but not 4.19) >>>> >>>> Craig >>>> >>>> [global] >>>> workgroup = XYZ >>>> security = user >>>> netbios name = red >>>> server string = red.example.net >>>> hostname lookups = no >>>> server role = standalone server >>>> >>>> interfaces = ixl0 lo0 >>>> bind interfaces only = yes >>>> >>>> load printers = no >>>> show add printer wizard = no >>>> time server = yes >>>> use mmap = yes >>>> >>>> dos charset = 850 >>>> unix charset = UTF-8 >>>> mangled names = no >>>> >>>> #log level = 3 >>>> #log file = /tmp/samba.log >>>> vfs objects = catia fruit streams_xattr zfsacl >>>> >>>> fruit:model = MacSamba >>>> fruit:resource = file >>>> fruit:metadata = netatalk >>>> fruit:nfs_aces = yes >>>> fruit:copyfile = no >>>> fruit:aapl = yes >>>> fruit:zero_file_id = yes >>>> >>>> inherit permissions = yes >>>> >>>> >>>> [Time Machine] >>>> path = /backups/mini >>>> read only = no >>>> guest ok = no >>>> writeable = yes >>>> browseable = yes >>>> fruit:resource = file >>>> fruit:time machine = yes >>>> valid users = backup-mini >>>> max disk size 512G >>>> >>>> hosts allow = 10.0.0.19 >>>> >> > [-- Attachment #2 --] <html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">I have never needed to explicitly set fruit:posix_rename, my Time Machine backups work completely fine without it (maybe it defaults to on now?). I have been using Samba 4.19 since the port came out too.<div><br></div><div>The only problem I encountered was with samba416 and fruit:zero_file_id, which severely borked TM shares, see <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269883">https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269883</a> for the gory details.</div><div><br></div><div>Since I don't have access to a Sonoma Mac on my network, I have used Windows instead to make a bunch of subdirectories in a Samba share with fruit:time machine enabled, but I can rename them at any level without issue.</div><div><br></div><div>-Dimitry</div><div><br></div><div><div><div><blockquote type="cite"><div>On 8 Sep 2024, at 14:10, David Chisnall <theraven@FreeBSD.org> wrote:</div><br class="Apple-interchange-newline"><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">I have opened <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=281360">https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=281360</a> to track this.<div><br></div><div>Reverting to samba416, I can back up in a new share, but I cannot back up to the share that had my existing backups in it. This may be a permissions issue or some problem withe metadata getting out of sync: moving the backupbundle from the old share to the new one does not fix it. Once I have a new complete backup, I will try comparing the metadata and seeing what the differences are.</div><div><br></div><div>So far, the main difference appears to be that the new backup creates a disk image bundle with 500 MiB bands whereas the old one was using 8 MiB bands. This change is presumably because APFS, unlike HFS+, supports holes in files, but I wonder if there’s a problem with Samba and very large directories? There were 47,486 bands in the old disk image.</div><div><br></div><div>David<br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 8 Sep 2024, at 12:48, David Chisnall <theraven@FreeBSD.org> wrote:</div><br class="Apple-interchange-newline"><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">A little bit more debugging, and a working hypothesis:<div><br></div><div>I have tried creating a new Time Machine share (empty directory) and pointing the Mac at it. On the first backup, it creates a `.com.apple.timemachine.supported-{uuid}` file and a `{uuid}-{date}.sparsebundle` directory. It appears to create a valid sparse bundle, but other posts suggest that it then renames this to `{computername}.backupbundle`. There is a Samba setting: `fruit:posix_rename` that is supposed to control this. It appears to fail at the point where it should do this rename. </div><div><br></div><div>If I mount the same share, I can reproduce this:</div><div><br></div><div><div>$ mkdir tmp</div><div>$ touch tmp/foo</div><div>$ mv tmp fmp</div><div>mv: rename tmp to fmp: Operation not supported</div></div><div><br></div><div>So it appears that something in the FreeBSD port of Samba has broken the ability to rename directories. </div><div><br></div><div>This appears to be recent breakage. Reverying to net/samba416 fixes this bit, at least, and I can now back up to a pristine share.</div><div><br></div><div>David<br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 7 Sep 2024, at 08:28, David Chisnall <theraven@FreeBSD.org> wrote:</div><br class="Apple-interchange-newline"><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div dir="auto"><div dir="ltr"></div><div dir="ltr">I believe this was broken by a macOS update around February. I’ve been trying to debug for a while. I’ve opened an Apple issue (FB14500950, for any Apple folks) but it shows up as few people reporting it. I posted on Mastodon and several people reported that Time Machine is broken and recommended Carbon Copy Cloner as an alternative. I would like to see it fixed, but it probably needs some more debugging by Apple folks. </div><div dir="ltr"><br></div><div dir="ltr">It stopped working for me with no changes on the server and I can reproduce the failures on two different Macs.</div><div dir="ltr"><br></div><div dir="ltr">Things I have tried:</div><div dir="ltr"><br></div><div dir="ltr"> - Upgrading Samba from 4.16 to 4.19</div><div dir="ltr"> - Upgrading FreeBSD from 13.x to 14.1</div><div dir="ltr"> - Setting the SMB timeout sysctls to larger values on macOS.</div><div dir="ltr"> - Turning up the SMB debug sysctls on macOS to see if there’s more info</div><div dir="ltr"> - Turning up the Samba logging level.</div><div dir="ltr"> - Verifying the backups</div><div dir="ltr"> - Watching smbinfo the server.</div><div dir="ltr"> - Updating macOS to the latest version</div><div dir="ltr"> - Connecting to the server with Finder and checking I can access files on the shares and that they have the right permissions.</div><div dir="ltr"><br></div><div dir="ltr">Samba doesn’t report any errors (I don’t know if there’s a way to force Samba to report permission-denied things).</div><div dir="ltr"><br></div><div dir="ltr">It appears that the Mac acquires a load of read-only locks and so does a lot of reads, but for some reason it appears to fail the first write. Even with a verify, it looks like it completes the verification bit but then fails to write to the plist file. </div><div dir="ltr"><br></div><div dir="ltr">With the increased debugging, I see this in the macOS Comsole:</div><div dir="ltr"><br></div><div dir="ltr"><span style="white-space: pre-wrap; caret-color: rgb(51, 51, 51); color: rgb(51, 51, 51); font-family: "SF Pro Text", "SF Pro Icons", "Helvetica Neue", Helvetica, Arial, sans-serif; letter-spacing: -0.374px; -webkit-text-size-adjust: 100%; background-color: rgb(255, 255, 255);">default 14:12:26.297714+0100 kernel smb2fs_smb_cmpd_create: smb2fs_smb_ntcreatex failed 13 default 14:12:26.301301+0100 kernel smb2fs_smb_cmpd_create: smb2fs_smb_ntcreatex failed 13 default 14:12:26.310563+0100 kernel smb2fs_smb_cmpd_query: smb2_smb_query_info (single request) failed 45 default 14:12:26.318319+0100 kernel smb2fs_smb_cmpd_query: smb2_smb_query_info (single request) failed 45 default 14:12:26.326850+0100 backupd -[DIStatFS initWithFileDescriptor:error:]: File system is smbfs default 14:12:26.542645+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 default 14:12:26.542682+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied default 14:12:26.543622+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 default 14:12:26.543657+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied default 14:12:26.543690+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 default 14:12:26.543697+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied default 14:12:26.543725+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80 default 14:12:26.543730+0100 kernel smbfs_vnop_access: TheRooT action = 0x80 denied default 14:12:26.544085+0100 kernel smbfs_vnop_access: 501 not authorized to access TheRooT : action = 0x80</span></div><div dir="ltr"><span style="white-space: pre-wrap; caret-color: rgb(51, 51, 51); color: rgb(51, 51, 51); font-family: "SF Pro Text", "SF Pro Icons", "Helvetica Neue", Helvetica, Arial, sans-serif; letter-spacing: -0.374px; -webkit-text-size-adjust: 100%; background-color: rgb(255, 255, 255);"><br></span></div><div dir="ltr"><span style="white-space: pre-wrap; caret-color: rgb(51, 51, 51); color: rgb(51, 51, 51); font-family: "SF Pro Text", "SF Pro Icons", "Helvetica Neue", Helvetica, Arial, sans-serif; letter-spacing: -0.374px; -webkit-text-size-adjust: 100%; background-color: rgb(255, 255, 255);">So it looks as if it is a permission issue. Maybe the mcOS SMB client has started using some bit of the protocol that Samba on FreeBSD doesn’t support for ACLs?</span></div><div dir="ltr"><br></div><div dir="ltr">David</div><div dir="ltr"><br></div><div dir="ltr"><blockquote type="cite">On 6 Sep 2024, at 22:48, Craig Leres <leres@freebsd.org> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><span>Last year you guys helped me get this to work with samba416. I recently tried to upgrade to samba419 and so far I'm unsuccessful. The error is "The backup disk image could not be created" and I'm running 14.1.</span><br><span></span><br><span>I'm using the same port build options with 4.16 and 4.19:</span><br><span></span><br><span> FAM</span><br><span> PYTHON3</span><br><span> QUOTAS</span><br><span> SYSLOG</span><br><span> UTMP</span><br><span> GSSAPI_BUILTIN</span><br><span> AVAHI</span><br><span> FRUIT</span><br><span></span><br><span>Having learned my lesson when I upgraded from 4.13 to 4.16, I removed the old backups from the zfs volume on the server before starting. I've also learned the rule that you need to delete and reattach the share on the mac side when you change the samba config.</span><br><span></span><br><span>Appended is the config that works with 4.16 (but not 4.19)</span><br><span></span><br><span> Craig</span><br><span></span><br><span>[global]</span><br><span> workgroup = XYZ</span><br><span> security = user</span><br><span> netbios name = red</span><br><span> server string = red.example.net</span><br><span> hostname lookups = no</span><br><span> server role = standalone server</span><br><span></span><br><span> interfaces = ixl0 lo0</span><br><span> bind interfaces only = yes</span><br><span></span><br><span> load printers = no</span><br><span> show add printer wizard = no</span><br><span> time server = yes</span><br><span> use mmap = yes</span><br><span></span><br><span> dos charset = 850</span><br><span> unix charset = UTF-8</span><br><span> mangled names = no</span><br><span></span><br><span> #log level = 3</span><br><span> #log file = /tmp/samba.log</span><br><span> vfs objects = catia fruit streams_xattr zfsacl</span><br><span></span><br><span> fruit:model = MacSamba</span><br><span> fruit:resource = file</span><br><span> fruit:metadata = netatalk</span><br><span> fruit:nfs_aces = yes</span><br><span> fruit:copyfile = no</span><br><span> fruit:aapl = yes</span><br><span> fruit:zero_file_id = yes</span><br><span></span><br><span> inherit permissions = yes</span><br><span></span><br><span></span><br><span>[Time Machine]</span><br><span> path = /backups/mini</span><br><span> read only = no</span><br><span> guest ok = no</span><br><span> writeable = yes</span><br><span> browseable = yes</span><br><span> fruit:resource = file</span><br><span> fruit:time machine = yes</span><br><span> valid users = backup-mini</span><br><span> max disk size 512G</span><br><span></span><br><span> hosts allow = 10.0.0.19</span><br><span></span><br></div></blockquote></div></div></blockquote></div><br></div></div></div></blockquote></div><br></div></div></div></blockquote></div><br></div></div></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?63540459-7D2E-4B70-962D-957550D294A3>