Date: Tue, 20 Feb 2018 12:08:18 +0000 From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 226064] Observing panic at ctl_free_beio+0x80 while running traffic on 100 iSCSI LUNs Message-ID: <bug-226064-8@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226064 Bug ID: 226064 Summary: Observing panic at ctl_free_beio+0x80 while running traffic on 100 iSCSI LUNs Product: Base System Version: CURRENT Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: manishk@chelsio.com CC: mav@FreeBSD.org Created attachment 190824 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=190824&action=edit core text file Observing panic while running traffic on 100 iSCSI LUNs (used zvols as LUNs) on the target machine. The panic stack trace: panic: Memory modified after free 0xfffff8016fadcb40(32) val=0 @ 0xfffff8016fadcb40 cpuid = 0 time = 1518462716 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe0065865370 kdb_backtrace() at kdb_backtrace+0x53/frame 0xfffffe0065865440 vpanic() at vpanic+0x268/frame 0xfffffe0065865510 kproc_shutdown() at kproc_shutdown/frame 0xfffffe0065865570 trash_ctor() at trash_ctor+0x6b/frame 0xfffffe00658655b0 uma_zalloc_arg() at uma_zalloc_arg+0x284/frame 0xfffffe00658656f0 bucket_alloc() at bucket_alloc+0x102/frame 0xfffffe0065865730 uma_zfree_arg() at uma_zfree_arg+0x647/frame 0xfffffe0065865820 uma_zfree() at uma_zfree+0x29/frame 0xfffffe0065865850 ctl_free_beio() at ctl_free_beio+0x80/frame 0xfffffe0065865880 ctl_complete_beio() at ctl_complete_beio+0x48/frame 0xfffffe00658658a0 ctl_be_block_dispatch_zvol() at ctl_be_block_dispatch_zvol+0x664/frame 0xfffffe00658659b0 ctl_be_block_worker() at ctl_be_block_worker+0x1fb/frame 0xfffffe0065865a40 taskqueue_run_locked() at taskqueue_run_locked+0x246/frame 0xfffffe0065865ab0 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe0065865b30 fork_exit() at fork_exit+0x145/frame 0xfffffe0065865bb0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0065865bb0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic ------------------------------------- Steps to recreate the issue: ------------------------------------- The target machine exposes 100 zvols as iSCSI LUNs. The Initiator discovers these LUNS, creates file system on them, and runs traffic (iozone). The target machine panics after few hours. OS: FreeBSD 12.0-CURRENT (svn r329019) I'm using GENERIC config file, with "nooptions VIMAGE" ==>Start the Target (FreeBSD) 1. Create 100 zvols # zpool create iscsi <slice/partition> # for ((i=0;i<100;i++)); do zfs create -V1G -o volmode=dev iscsi/d$i; done 2. configure the network interface to be used (Target_IP_Address) 2. /etc/ctl.conf file: portal-group pg0 { discovery-auth-group no-authentication listen <Target_IP_Address> } target iqn.2016-11.com.xyz.abc:0 { auth-group no-authentication portal-group pg0 lun 0 { path /dev/zvol/iscsi/d0 size 1G option vendor "foo" option product "bar" option revision "d0" } } . . 100 targets with 1 lun each 3. start the target daemon # service ctld onestart ==>Initiator (Linux) 1. login/discover the target # iscsiadm -m discovery -t st -p <Target_IP_Address> -l 2. List all Luns # lsscsi 3. Format and create file system ONLY on all discovered luns (TAKE CARE TO NOT TOUCH OTHER LUNS). # mkfs.ext3 /dev/sdxx // for all discovered LUNs 4. mount all the devices formatted in the above step to different mount points # mount /dev/sdxx /mnt/iscsiyy // for all the luns formatted in the above step 5. Run traffic on all luns using iozone # cd /mnt/iscsiyy # iozone -a -I -+d -g 512m & ==> Target machine panics after 3 to 10 hours. -- You are receiving this mail because: You are the assignee for the bug.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-226064-8>