Date: Mon, 14 Nov 2011 10:59:14 +0100 From: Peter Maloney <peter.maloney@brockmann-consult.de> To: freebsd-current@freebsd.org Subject: Re: samba+zfs Message-ID: <4EC0E672.2030700@brockmann-consult.de> In-Reply-To: <alpine.BSF.2.00.1111140253450.1814@sunsaturn.com> References: <alpine.BSF.2.00.1110272039500.50739@sunsaturn.com> <CAGH67wRZZx0hG9ug2k-5ohCOPJ9sZOU9iFVKg7hv9WM=R761GA@mail.gmail.com> <alpine.BSF.2.00.1111080259270.89703@sunsaturn.com> <alpine.BSF.2.00.1111080328010.89703@sunsaturn.com> <CALM%2B6aJkF=CFq8LA3FrSMYo8La-8txK4h2p4yZtdHshskBU6Vw@mail.gmail.com> <alpine.BSF.2.00.1111082015260.93923@sunsaturn.com> <CALM%2B6aJiM%2BNfikax9vNiYzBJqo3B8WLEXiZgrjUUQa9ngCQaKg@mail.gmail.com> <B4FEAF5B-52AC-40E5-90A3-7BD060BB7A73@gsoft.com.au> <F8AEDB62-4C76-4D95-8CBA-6C58B54C1965@gsoft.com.au> <CAGH67wRJkN%2BsDhxxMVSZx3RXQJm5xyyfW9iwiY1FrkFxLu%2BTWQ@mail.gmail.com> <82C85C01-62C4-4E75-B3F2-59D703CA5D78@gsoft.com.au> <CALM%2B6aK3PgdPNZJEHCdday84tH_7P9Ug=_yk=PfxcFOV6ejEzA@mail.gmail.com> <alpine.BSF.2.00.1111120444440.65294@sunsaturn.com> <alpine.BSF.2.00.1111140253450.1814@sunsaturn.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Dan,
I may have had the same problem and solved it. I'm not quite sure at all
it is related though.
Try adding these 2 lines to your [global] section of your smb.conf on
the samba server:
strict locking = no
blocking locks = no
I don't know if the above will cause some data integrity penalty
associated with crashes, power failures, etc. with synchronous writes,
or people opening and writing the same file.
Details about my case:
I have a tight security ZFS file server that uses NFS only.
I have a second (virtual machine) file server with lots of user accounts
(the reason it is a separate server) that actually has no files to
serve, but uses the NFS mounts to share files over samba.
Without the locking options set to "no", is a small amount of activity,
maybe only with more than 1 client on the files, it seems like the samba
share is totally locked up when trying to do a listing of a directory
with more than 5 or so files in it.
And *please *confirm that you have checked the io load % on your disks
using gstat when doing this writing. If you don't do that, it could even
be as simple as a single disk in an array having very poor access times
(such as what would happen on a failing consumer grade disk with bad media).
For example:
http://fsk141.com/slow-zfs-zpool
he ends with "I wish I would have ran an iostat before trying all the
things that I did :("
My own example:
I had a bad disk in a 16 disk array. It caused boot to take so long that
I removed the disk and rebooted again instead of waiting. Scrubs would
go 51 MB/s with the disk ONLINE, and 761 MB/s with the disk OFFLINE.
Doing any testing would show the disk at 100-128% load with all other
disks at 0%. Resilvering had similar results. Replacing the disk made
everything perfectly normal again.
Peter
On 11/14/2011 09:56 AM, Dan The Man wrote:
>
>
> Running tcpdump to trace what samba is doing so maybe someone can give
> some insight, lan interface is sk0.
>
>
> 02:52:34.347357 IP (tos 0x0, ttl 64, id 56121, offset 0, flags [none],
> proto TCP (6), length 1500)
> asterisk.microsoft-ds > desktop.58858: Flags [.], cksum 0x5e3f
> (correct), seq 105687574:105689034, ack 63894978, win 256, length
> 1460SMB-over-TCP packet:(raw data or continuation?)
>
> 02:52:34.347361 IP (tos 0x0, ttl 64, id 56122, offset 0, flags [none],
> proto TCP (6), length 1500)
> asterisk.microsoft-ds > desktop.58858: Flags [.], cksum 0xbe99
> (correct), seq 105689034:105690494, ack 63894978, win 256, length
> 1460SMB-over-TCP packet:(raw data or continuation?)
>
> 02:52:34.347365 IP (tos 0x0, ttl 64, id 56123, offset 0, flags [none],
> proto TCP (6), length 1500)
> asterisk.microsoft-ds > desktop.58858: Flags [.], cksum 0x0b31
> (correct), seq 105690494:105691954, ack 63894978, win 256, length
> 1460SMB-over-TCP packet:(raw data or continuation?)
>
> 02:52:34.347369 IP (tos 0x0, ttl 64, id 56124, offset 0, flags [none],
> proto TCP (6), length 1500)
> asterisk.microsoft-ds > desktop.58858: Flags [.], cksum 0xaee6
> (correct), seq 105691954:105693414, ack 63894978, win 256, length
> 1460SMB-over-TCP packet:(raw data or continuation?)
>
> 02:52:34.347372 IP (tos 0x0, ttl 64, id 56125, offset 0, flags [none],
> proto TCP (6), length 1500)
> asterisk.microsoft-ds > desktop.58858: Flags [.], cksum 0xcff5
> (correct), seq 105693414:105694874, ack 63894978, win 256, length
> 1460SMB-over-TCP packet:(raw data or continuation?)
>
> 02:52:34.347376 IP (tos 0x0, ttl 64, id 56126, offset 0, flags [none],
> proto TCP (6), length 1500)
> asterisk.microsoft-ds > desktop.58858: Flags [.], cksum 0xd893
> (correct), seq 105694874:105696334, ack 63894978, win 256, length
> 1460SMB-over-TCP packet:(raw data or continuation?)
>
> We get a ton of these, my mapped samba drive on Z: becomes nearly
> unresponsive after i start transferring things through it, yet I can
> jump on Y: drive which is NFS mount to same interface on machine and
> everything is fine and responsive....
>
>
> Dan.
>
>
> --
> Dan The Man
> CTO/ Senior System Administrator
> Websites, Domains and Everything else
> http://www.SunSaturn.com
> Email: Dan@SunSaturn.com
>
> On Sat, 12 Nov 2011, Dan The Man wrote:
>
>>
>>
>> Well been running a week now and problems again. 3 3 terrabyte drives
>> are @85% with compression enabled, i have to wonder if that is part
>> of the problem.
>>
>>
>>
>> Dan.
>>
>>
>>
>> --
>> Dan The Man
>> CTO/ Senior System Administrator
>> Websites, Domains and Everything else
>> http://www.SunSaturn.com
>> Email: Dan@SunSaturn.com
>>
>> On Wed, 9 Nov 2011, Kurt Touet wrote:
>>
>>> On Wed, Nov 9, 2011 at 1:07 AM, Daniel O'Connor
>>> <doconnor@gsoft.com.au> wrote:
>>>>
>>>> On 09/11/2011, at 17:32, Garrett Cooper wrote
>>>>>> dd's of large files (spooled backups going to tape) to /dev/null
>>>>>> are as slow as Samba.
>>>>>
>>>>> - Dedupe?
>>>>
>>>> Nope.
>>>>
>>>>> - Compression?
>>>>
>>>> On the mail spool & ports, but not on the tape spool.
>>>>
>>>>> - How much RAM?
>>>>
>>>> 8GB.
>>>>
>>>>> - What debug options do you have enabled in the kernel?
>>>>
>>>> It is 8.2-GENERIC so.. no WITNESS (for example)
>>>>
>>>>> I've been noticing a slowdown in some respects with NFS/SMB, but I
>>>>> suspected it was because I have an re(4) based NIC. ZFS has also
>>>>> wired
>>>>> down a lot of my system memory for the L2ARC…
>>>>
>>>>
>>>> re isn't great but I wouldn't expect it to slow down over time..
>>>> Unless bounce buffers got used more and more or something.
>>>>
>>>> I have an em0 card in this system - but in any case it is slow
>>>> locally (i.e. dd a large file with 64k block size).
>>>>
>>>> --
>>>> Daniel O'Connor software and network engineer
>>>> for Genesis Software - http://www.gsoft.com.au
>>>> "The nice thing about standards is that there
>>>> are so many of them to choose from."
>>>> -- Andrew Tanenbaum
>>>> GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
>>>>
>>>>
>>>
>>> Right now (while experience slow writes via samba+zfs) this is general
>>> read speed off a 4 x 1.5TB sata2 raidz1:
>>>
>>> # dd if=test.file of=/dev/null
>>> 13753502+1 records in
>>> 13753502+1 records out
>>> 7041793036 bytes transferred in 100.020897 secs (70403218 bytes/sec)
>>>
>>> That's not in the same ball park of slow writes, but it is below what
>>> I expect for reads.
>>>
>>> My setup is a little odd: 4x1.5tb raidz sata2 on mobo + 2 x 2tb
>>> mirror on sata1 pci controller, zfs v28, stable/9 r227357, amd x4 810
>>> 2.6ghz, 4gb ram, no dedupe, no compression, daily snapshots saved for
>>> 7 days
>>>
>>> The above file read was stored before the 2 x 2tb mirror addition, so
>>> it was a solely read off the sata2 mobo ports. Reading off of
>>> something more recent (and split amongst both raidz1 and mirror
>>> vdevs):
>>>
>>> # dd if=test2.file of=/dev/null
>>> 9154715+1 records in
>>> 9154715+1 records out
>>> 4687214153 bytes transferred in 82.963181 secs (56497522 bytes/sec)
>>>
>>> This is, again, seems slower than usual, but not as terrible as the
>>> write speeds that I've been seeing via samba.
>>
>
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
--
--------------------------------------------
Peter Maloney
Brockmann Consult
Max-Planck-Str. 2
21502 Geesthacht
Germany
Tel: +49 4152 889 300
Fax: +49 4152 889 333
E-mail: peter.maloney@brockmann-consult.de
Internet: http://www.brockmann-consult.de
--------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4EC0E672.2030700>