Meyer <dinoex@FreeBSD.org> ---
I see no difference serving from ZFS or UFS
I can not reproduce the problem in the VM.

1)
hardware 1, jail, serving from ZFS:
server.event-handler =3D "freebsd-kqueue"
server.network-backend =3D "writev"

changes to this config lines did not effect the problem.

CPU: Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz (2133.42-MHz K8-class =
CPU)
  Origin=3D"GenuineIntel"  Id=3D0x106a5  Family=3D0x6  Model=3D0x1a  Steppi=
ng=3D5
=20
Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,=
MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
=20
Features2=3D0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDC=
M,DCA,SSE4.1,SSE4.2,POPCNT>
  AMD Features=3D0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=3D0x1<LAHF>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID
  TSC: P-state invariant, performance statistics


igb0: <Intel(R) PRO/1000 82576> port 0xef80-0xef9f mem
0xfafe0000-0xfaffffff,0xfafc0000-0xfafdffff,0xfaf9c000-0xfaf9ffff irq 28 at
device 0.0 on pci1
igb0: EEPROM V1.2-3
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 4 RX queues 4 TX queues
igb0: Using MSI-X interrupts with 5 vectors
igb0: Ethernet address: 00:25:90:**:**:**
igb0: netmap queues/slots: TX 4/1024, RX 4/1024


# sysctl kern.ipc.tls | grep -v ': 0'
kern.ipc.tls.ifnet.permitted: 1
kern.ipc.tls.sw.gcm: 316
kern.ipc.tls.max_reclaim: 1024
kern.ipc.tls.sw_buffer_cache: 1
kern.ipc.tls.cbc_enable: 1
kern.ipc.tls.enable: 1
kern.ipc.tls.ifnet_max_rexmit_pct: 2
kern.ipc.tls.maxlen: 16384
kern.ipc.tls.stats.ocf.separate_output: 320188
kern.ipc.tls.stats.ocf.inplace: 305320885
kern.ipc.tls.stats.ocf.tls13_chacha20_encrypts: 108010
kern.ipc.tls.stats.ocf.tls13_chacha20_decrypts: 54048
kern.ipc.tls.stats.ocf.tls13_gcm_encrypts: 304731247
kern.ipc.tls.stats.ocf.tls13_gcm_decrypts: 152629728
kern.ipc.tls.stats.ocf.tls12_chacha20_encrypts: 17698
kern.ipc.tls.stats.ocf.tls12_chacha20_decrypts: 14882
kern.ipc.tls.stats.ocf.tls12_gcm_encrypts: 784118
kern.ipc.tls.stats.ocf.tls12_gcm_decrypts: 555362
kern.ipc.tls.stats.failed_crypto: 1332
kern.ipc.tls.stats.corrupted_records: 75
kern.ipc.tls.stats.active: 316
kern.ipc.tls.stats.enable_calls: 145561272
kern.ipc.tls.stats.offload_total: 145561272
kern.ipc.tls.stats.threads: 8


2)
hardware 2, no jail, serving UFS on gmirror

CPU: Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz (3400.20-MHz K8-class CPU)
  Origin=3D"GenuineIntel"  Id=3D0x306c3  Family=3D0x6  Model=3D0x3c  Steppi=
ng=3D3
=20
Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,=
MCA,CMOV,PAT,PSE36,CLFLUS
H,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
=20
Features2=3D0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSS=
E3,SDBG,FMA,CX16,xTPR,PDC
M,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16=
C,RDRAND>
  AMD Features=3D0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=3D0x21<LAHF,ABM>
  Structured Extended
Features=3D0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
  Structured Extended
Features3=3D0x9c000600<MCUOPT,MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=3D0x1<XSAVEOPT>
  VT-x: Basic Features=3D0xda0400<SMM,INS/OUTS,TRUE>
        Pin-Based Controls=3D0x7f<ExtINT,NMI,VNMI,PreTmr>
        Primary Processor
Controls=3D0xfff9fffe<INTWIN,TSCOff,HLT,INVLPG,MWAIT,RDPMC,RDTSC,CR3-LD,CR3-
ST,CR8-LD,CR8-ST,TPR,NMIWIN,MOV-DR,IO,IOmap,MTF,MSRmap,MONITOR,PAUSE>
        Secondary Processor
Controls=3D0x7cff<APIC,EPT,DT,RDTSCP,x2APIC,VPID,WBINVD,UG,PAUSE-loop,RDRAN=
D,INVPCID,VMFUNC,VMCS>
        Exit Controls=3D0xda0400<PAT-LD,EFER-SV,PTMR-SV>
        Entry Controls=3D0xda0400
        EPT Features=3D0x6334141<XO,PW4,UC,WB,2M,1G,INVEPT,AD,single,all>
        VPID Features=3D0xf01<INVVPID,individual,single,all,single-globals>
  TSC: P-state invariant, performance statistics


igb0: promiscuous mode disabled
igb0: <Intel(R) I210 (Copper)> port 0xd000-0xd01f mem
0xf7100000-0xf717ffff,0xf7180000-0xf7183fff irq 17 at device 0.0 on pci3
igb0: attach_pre capping queues at 4
igb0: EEPROM V3.16-0 eTrack 0x800004d7
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: msix_init qsets capped at 4
igb0: intr CPUs: 4 queue msgs: 4 admincnt: 1
igb0: Using 4 RX queues 4 TX queues
igb0: attempting to allocate 5 MSI-X vectors (5 supported)
igb0: using IRQs 34-38 for MSI-X
igb0: Using MSI-X interrupts with 5 vectors
igb0: allocated for 4 tx_queues
igb0: allocated for 4 rx_queues
igb0: bpf attached
igb0: Ethernet address: 00:25:90:**:**:**
igb0: netmap queues/slots: TX 4/1024, RX 4/1024
igb0: Link is up 1000 Mbps Full Duplex
igb0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from igb0 (ifp
0xfffff80001d50800), ignoring.
igb0: link state changed to DOWN
igb0: Link is up 1000 Mbps Full Duplex
igb0: link state changed to UP


# sysctl kern.ipc.tls | grep -v ': 0'
kern.ipc.tls.ifnet.permitted: 1
kern.ipc.tls.max_reclaim: 1024
kern.ipc.tls.sw_buffer_cache: 1
kern.ipc.tls.cbc_enable: 1
kern.ipc.tls.enable: 1
kern.ipc.tls.ifnet_max_rexmit_pct: 2
kern.ipc.tls.maxlen: 16384
kern.ipc.tls.stats.ocf.separate_output: 2114
kern.ipc.tls.stats.ocf.inplace: 12982528
kern.ipc.tls.stats.ocf.tls13_chacha20_encrypts: 232
kern.ipc.tls.stats.ocf.tls13_chacha20_decrypts: 191
kern.ipc.tls.stats.ocf.tls13_gcm_encrypts: 12984410
kern.ipc.tls.stats.ocf.tls13_gcm_decrypts: 6557814
kern.ipc.tls.stats.enable_calls: 3204927
kern.ipc.tls.stats.offload_total: 3204927
kern.ipc.tls.stats.threads: 4

3)
bhyve VM with vtnet0, no problems

vtnet0: <VirtIO Networking Adapter> on virtio_pci5
vtnet0: Ethernet address: 00:a0:98:**:**:**
vtnet0: netmap queues/slots: TX 1/1024, RX 1/512
vtnet0: link state changed to UP


# sysctl kern.ipc.tls | grep -v ': 0'
kern.ipc.tls.ifnet.permitted: 1
kern.ipc.tls.max_reclaim: 1024
kern.ipc.tls.sw_buffer_cache: 1
kern.ipc.tls.cbc_enable: 1
kern.ipc.tls.enable: 1
kern.ipc.tls.ifnet_max_rexmit_pct: 2
kern.ipc.tls.maxlen: 16384

--=20
You are receiving this mail because:
You are the assignee for the bug.=