Date: Wed, 19 Jul 2006 09:34:26 -0400 From: Randall Stewart <rrs@cisco.com> To: Pawel Worach <pawel.worach@gmail.com> Cc: freebsd-net@freebsd.org Subject: Re: SCTP Message-ID: <44BE34E2.7070603@cisco.com> In-Reply-To: <d227e09e0607181323q18e53947p942c944602c43cfe@mail.gmail.com> References: <44BB7A92.9080008@cisco.com> <d227e09e0607181323q18e53947p942c944602c43cfe@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Pawel:
I see at least one thing wrong with the sctp_sendmsg()
code... I just recently added the iov.... and the
order of where bad:/ bad2:/ bad1: goes is wrong..
Now, the MAC stuff I have never enabled (at least
I don't think so).. and I see that in this trace
it seems the MAC stuff is calling to deallocate
the socket directly... I am not sure if the
crash is related to the wrong bad calls.. which
would do a free() when it should not on the iov..
that can't be good.. but also not sure of the
deallocate() stuff...
The bad stuff is easy to fix.. and I will
get a new patch prepared.. (I also will see if
I can't update to current again.. and thus
eliminate your syscall conflict)..
But I want to look a bit into this mac_destroy_socket()
path...
R
Pawel Worach wrote:
> On 7/17/06, Randall Stewart <rrs@cisco.com> wrote:
>
>> All:
>>
>> Just a friendly reminder/prod... if you have started
>> testing SCTP.. thats great (any feedback?)..
>> and if you have not .. please do so :-D
>
>
> Hi,
>
> I played around a bit with NetPIPE, FreeBSD-CURRENT in one end and
> Linux 2.6.17 in the other over a gigabit crossover cable network, 1500
> MTU. FreeBSD crashes after a while. I do have MAC enabled (no policy
> modules loaded at the time), it looks like it is involved. I think I
> can reproduce this, made it happen on both attempts.
>
> For the record, I modified the patch a bit to make it compile, the
> syscalls numbers collide with new threading syscalls added recently,
> so I moved the thr syscalls up a notch. And I removed this #ifdef MAC
> part of the patch due to duplicate sctp_bad labels.
>
> +#ifdef MAC
> +sctp_bad:
> +#endif
> + sctp_bad:
> + free(iov, M_IOV);
>
> Any more info I can provide ?
>
> ~/sctp/np> ./NPsctp -h 192.168.10.1
> ...
> 68: 16384 bytes 71 times --> 179.87 Mbps in 694.94 usec
> 69: 16387 bytes 71 times --> 178.78 Mbps in 699.33 usec
> 70: 24573 bytes 71 times --> 198.43 Mbps in 944.80 usec
> 71: 24576 bytes 70 times --> 199.18 Mbps in 941.35 usec
> 72: 24579 bytes 70 times --> 198.82 Mbps in 943.19 usec
> 73: 32765 bytes 35 times --> 210.05 Mbps in 1190.07 usec
> 74: 32768 bytes 42 times --> 208.48 Mbps in 1199.15 usec
> 75: 32771 bytes 41 times --> 208.00 Mbps in 1202.03 usec
> 76: 49149 bytes 41 times --> 234.43 Mbps in 1599.55 usec
> 77: 49152 bytes 41 times --> 300.20 Mbps in 1249.17 usec
> 78: 49155 bytes 53 times --> 299.66 Mbps in 1251.51 usec
> 79: 65533 bytes 26 times --> 4.77 Mbps in 104844.52 usec
> 80: 65536 bytes 3 times --> 3.70 Mbps in 135258.48 usec
> 81: 65539 bytes 3 times --> 3.70 Mbps in 135257.16 usec
> 82: 98301 bytes 3 times --> 7.36 Mbps in 101946.00 usec
> 83: 98304 bytes 3 times --> 7.36 Mbps in 101923.51 usec
> 84: 98307 bytes 3 times --> 7.36 Mbps in 101945.48 usec
> 85: 131069 bytes 3 times --> [stalls here]
>
> then a couple of seconds later...
>
> Fatal trap 12: page fault while in kernel mode
> fault virtual address = 0x0
> fault code = supervisor write, page not present
> instruction pointer = 0x20:0xc06a7e16
> stack pointer = 0x28:0xd35e5174
> frame pointer = 0x28:0xd35e5174
> code segment = base 0x0, limit 0xfffff, type 0x1b
> = DPL 0, pres 1, def32 1, gran 1
> processor eflags = interrupt enabled, resume, IOPL = 0
> current process = 12 (swi1: net)
> trap number = 12
> panic: page fault
> KDB: stack backtrace:
> kdb_backtrace(c078488a,c07e2500,c07790c0,d35e5028,100,...) at
> kdb_backtrace+0x2e
> panic(c07790c0,c079de93,c2466a70,1,1,...) at panic+0xb7
> trap_fatal(d35e5134,0,2,8,e5df6f6e,...) at trap_fatal+0x342
> trap_pfault(d35e5134,0,0,0,0,...) at trap_pfault+0x245
> trap(8,ffff0028,7fff0028,c104db80,0,...) at trap+0x3e3
> calltrap() at calltrap+0x5
> --- trap 0xc, eip = 0xc06a7e16, esp = 0xd35e5174, ebp = 0xd35e5174 ---
> mac_labelzone_dtor(0,14,0,0,0,...) at mac_labelzone_dtor+0x6
> uma_zfree_arg(c104db80,0,0,d35e51d0,c06acfc4,...) at uma_zfree_arg+0x2f
> mac_labelzone_free(0) at mac_labelzone_free+0x22
> mac_socket_label_free(0,c2ad4000,d35e5200,c0587da8,c2ad4000,...) at
> mac_socket_label_free+0x94
> mac_destroy_socket(c2ad4000,40,0,c2ad4000,4,...) at mac_destroy_socket+0x18
> sodealloc(c2ad4000,c2ad4000,0,0,4,...) at sodealloc+0x168
> sofree(c2ad4000,0,0,0,c10372c8,...) at sofree+0x311
> sctp_inpcb_free(c2c98000,0,0,d35e52b4,c060c90d,...) at
> sctp_inpcb_free+0x10d6
> sctp_free_assoc(c2c98000,c2cad958,0,c2cafcd0,d35e534c,...) at
> sctp_free_assoc+0x1a5b
> sctp_handle_shutdown_complete(c2cf3830,c2cad958,c2cafcd0,d35e534c,c0754bbe,...)
>
> at sctp_handle_shutdown_complete+0x228
> sctp_process_control(c2cea500,14,d35e5bb8,24,c2cf3824,...) at
> sctp_process_control+0x1388
> sctp_common_input_processing(d35e5c30,14,20,24,c2cf3824,...) at
> sctp_common_input_processing+0x87
> sctp_input(c2cea500,14,c255c800,1,0,...) at sctp_input+0x383
> ip_input(c2cea500,d35e5c88,c0553c65,8,0,...) at ip_input+0x70c
> netisr_processqueue(c07e75b8,c2467870,c2467870,c24668d0,d35e5ce4,...)
> at netisr_processqueue+0xe9
> swi_net(0,c2467870,80246,b9669622,c2467870,...) at swi_net+0x12f
> ithread_execute_handlers(c24668d0,c2463500,c24668d0,c2467870,c24668d0,...)
> at ithread_execute_handlers+0x188
> ithread_loop(c2433ad0,d35e5d38,0,0,c2433ad0,...) at ithread_loop+0x76
> fork_exit(c051d900,c2433ad0,d35e5d38) at fork_exit+0x7f
> fork_trampoline() at fork_trampoline+0x8
> --- trap 0x1, eip = 0, esp = 0xd35e5d6c, ebp = 0 ---
> Uptime: 27m28s
> Physical memory: 502 MB
> Dumping 68 MB: 53 37 21 5
>
> #0 doadump () at pcpu.h:166
> 166 pcpu.h: No such file or directory.
> in pcpu.h
> (kgdb) bt
> #0 doadump () at pcpu.h:166
> #1 0xc053c0b4 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
> #2 0xc053c42d in panic (fmt=0xc07790c0 "%s")
> at /usr/src/sys/kern/kern_shutdown.c:565
> #3 0xc074a2d2 in trap_fatal (frame=0xd35e5134, eva=0)
> at /usr/src/sys/i386/i386/trap.c:869
> #4 0xc0749f65 in trap_pfault (frame=0xd35e5134, usermode=0, eva=0)
> at /usr/src/sys/i386/i386/trap.c:778
> #5 0xc0749ab3 in trap (frame=
> {tf_fs = 8, tf_es = -65496, tf_ds = 2147418152, tf_edi =
> -1056646272, tf_esi = 0, tf_ebp = -748793484, tf_isp = -748793504,
> tf_ebx = 0, tf_edx = 0, tf_ecx = 4, tf_eax = 0, tf_trapno = 12, tf_err
> = 2, tf_eip = -1066762730, tf_cs = 32, tf_eflags = 66178, tf_esp =
> -748793432, tf_ss = -1066463889})
> at /usr/src/sys/i386/i386/trap.c:463
> #6 0xc0738cfa in calltrap () at /usr/src/sys/i386/i386/exception.s:138
> #7 0xc06a7e16 in mac_labelzone_dtor (mem=0x0, size=20, arg=0x0)
> at /usr/src/sys/security/mac/mac_label.c:74
> #8 0xc06f0d6f in uma_zfree_arg (zone=0xc104db80, item=0x0, udata=0x0)
> at /usr/src/sys/vm/uma_core.c:2263
> #9 0xc06a7e72 in mac_labelzone_free (label=0x0) at uma.h:303
> #10 0xc06acfc4 in mac_socket_label_free (label=0x0)
> at /usr/src/sys/security/mac/mac_socket.c:151
> #11 0xc06ad088 in mac_destroy_socket (socket=0xc2ad4000)
> ---Type <return> to continue, or q <return> to quit---
> at /usr/src/sys/security/mac/mac_socket.c:168
> #12 0xc0587da8 in sodealloc (so=0xc2ad4000)
> at /usr/src/sys/kern/uipc_socket.c:291
> #13 0xc0588971 in sofree (so=0xc2ad4000) at
> /usr/src/sys/kern/uipc_socket.c:592
> #14 0xc0604986 in sctp_inpcb_free (inp=0xc2c98000, immediate=0)
> at /usr/src/sys/netinet/sctp_pcb.c:2582
> #15 0xc060817b in sctp_free_assoc (inp=0xc2c98000, stcb=0xc2cad958,
> from_inpcbfree=0) at /usr/src/sys/netinet/sctp_pcb.c:3896
> #16 0xc0617b58 in sctp_handle_shutdown_complete (cp=0xc2cf3830,
> stcb=0xc2cad958, net=0x0) at /usr/src/sys/netinet/sctp_input.c:2500
> #17 0xc061a7d8 in sctp_process_control (m=0xc2cea500, iphlen=20,
> offset=0xd35e5bb8, length=36, sh=0xc2cf3824, ch=0xc2cf3830,
> inp=0xc2c98000, stcb=0xc2cad958, netp=0xd35e5bd0,
> fwd_tsn_seen=0xd35e5b98)
> at /usr/src/sys/netinet/sctp_input.c:4267
> #18 0xc061ad87 in sctp_common_input_processing (mm=0xd35e5c30, iphlen=20,
> offset=32, length=36, sh=0xc2cf3824, ch=0xc2cf3830, inp=0xc2c98000,
> stcb=0xc2cad958, net=0xc2cafcd0, ecn_bits=2 '\002')
> at /usr/src/sys/netinet/sctp_input.c:4583
> #19 0xc061b5e3 in sctp_input (m=0xc2cea500, off=20)
> at /usr/src/sys/netinet/sctp_input.c:4994
> #20 0xc05ec1ec in ip_input (m=0xc2cea500)
> at /usr/src/sys/netinet/ip_input.c:658
> #21 0xc05d2de9 in netisr_processqueue (ni=0xc07e75b8)
> ---Type <return> to continue, or q <return> to quit---
> at /usr/src/sys/net/netisr.c:236
> #22 0xc05d305f in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:349
> #23 0xc051d808 in ithread_execute_handlers (p=0xc24668d0, ie=0xc2463500)
> at /usr/src/sys/kern/kern_intr.c:662
> #24 0xc051d976 in ithread_loop (arg=0xc2433ad0)
> at /usr/src/sys/kern/kern_intr.c:745
> #25 0xc051c38f in fork_exit (callout=0xc051d900 <ithread_loop>, arg=0x0,
> frame=0x0) at /usr/src/sys/kern/kern_fork.c:822
> #26 0xc0738d5c in fork_trampoline () at
> /usr/src/sys/i386/i386/exception.s:199
>
--
Randall Stewart
NSSTG - Cisco Systems Inc.
803-345-0369 <or> 815-342-5222 (cell)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44BE34E2.7070603>