Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 1 Aug 2005 17:00:33 GMT
From:      Mark Blackman <mark@exonetric.com>
To:        freebsd-bugs@FreeBSD.org
Subject:   Re: kern/84389: 4.11-STABLE stuck in kernel-space?
Message-ID:  <200508011700.j71H0X6s064092@freefall.freebsd.org>

next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/84389; it has been noted by GNATS.

From: Mark Blackman <mark@exonetric.com>
To: Marc G.Fournier <scrappy@hub.org>
Cc: freebsd-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Subject: Re: kern/84389: 4.11-STABLE stuck in kernel-space?
Date: Mon, 1 Aug 2005 17:57:08 +0100

 On 31 Jul 2005, at 22:55, Marc G. Fournier wrote:
 
 >
 > 'k, I don't know how much RAM you have on this machine ... but, on  
 > all of ours, we have 4G, and to fix *our* vnode issue, I have to  
 > build the kernel/world with:
 
 1GB ram.
 
 >
 > CFLAGS= -O -mpentium -pipe -g -DKVA_PAGES=512
 > COPTFLAGS= -O -mpentium -pipe -DKVA_PAGES=512
 >
 > and I have /etc/sysctl.conf setup with:
 >
 > kern.maxvnodes=522240
 >
 > The fact that its dying at about the same time each day, I'm going  
 > to guess that its something like the 'find' that runs for the  
 > various periodic "security" checks that is sucking back the vnodes  
 > for you ...
 >
 > This is all speculation, but this sounds like exactly everythign I  
 > went through when I started to load up machines with jails :)
 
 It appears we are in a very similar line of business. :)
 
 Is there any data I can provide to pin this down. It didn't happen  
 today, FWIW.
 
  From my reading of the core dump, it looks like it might have stuck  
 on a
 file write in a newly configured jail. It was a postfix 'postdrop'  
 process
 where the DDB froze it. Assuming it was actually stuck there, then that
 write was the bad guy.
 
 Cheers,
 Mark
 
 >
 > hope this helps ...
 >
 >  On Sun, 31 Jul 2005, Mark Blackman wrote:
 >
 >
 >>
 >>
 >>> Number:         84389
 >>> Category:       kern
 >>> Synopsis:       4.11-STABLE stuck in kernel-space?
 >>> Confidential:   no
 >>> Severity:       serious
 >>> Priority:       medium
 >>> Responsible:    freebsd-bugs
 >>> State:          open
 >>> Quarter:
 >>> Keywords:
 >>> Date-Required:
 >>> Class:          sw-bug
 >>> Submitter-Id:   current-users
 >>> Arrival-Date:   Sun Jul 31 14:00:22 GMT 2005
 >>> Closed-Date:
 >>> Last-Modified:
 >>> Originator:     Mark Blackman
 >>> Release:        4.11-Stable
 >>> Organization:
 >>> Environment:
 >>>
 >> FreeBSD varadero.exonetric.net 4.11-STABLE FreeBSD 4.11-STABLE #2:  
 >> Sat Apr 23 22:20:21 BST 2005     root@varadero.exonetric.net:/usr/ 
 >> obj/usr/src/sys/MAIN-NOSMP  i386
 >>
 >>> Description:
 >>>
 >> System is unresponsive to any network or console userland inputs,  
 >> however it does respond to ping and can break into ddb via serial  
 >> console. I took a panic and rebooted. I have a full debug kernel  
 >> and the full core dump. I've pasted in a full backtrace for  
 >> reference and can answer any question about this core dump. Looks  
 >> to me like some kind of issue
 >> on vnode file write. As a curious side note, this condition has  
 >> manifested itself at approximately if not exactly the same time  
 >> every day when it has happened (09:20-09:25).
 >> This is the first reoccurence after several months of dormancy.
 >>
 >> The key features of this system are several jails and vnodes (~15  
 >> each).
 >>
 >> backtrace full follows..
 >>
 >> #0  dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
 >>        error = 0
 >> #1  0xc01b940b in boot (howto=256) at /usr/src/sys/kern/ 
 >> kern_shutdown.c:316
 >>        howto = 256
 >> #2  0xc01b9849 in panic (fmt=0xc0330744 "from debugger") at /usr/ 
 >> src/sys/kern/kern_shutdown.c:595
 >>        fmt = 0xc0330744 "from debugger"
 >>        bootopt = 256
 >>        buf = "from debugger", '\000' <repeats 242 times>
 >> #3  0xc0144739 in db_panic (addr=-1070466719, have_addr=0,  
 >> count=-1, modif=0xe3711ab0 "")
 >>    at /usr/src/sys/ddb/db_command.c:435
 >> No locals.
 >> #4  0xc01446d9 in db_command (last_cmdp=0xc0387eb8,  
 >> cmd_table=0xc0387cf8, aux_cmd_tablep=0xc03aeb18)
 >>    at /usr/src/sys/ddb/db_command.c:333
 >>        cmd_table = (struct command *) 0x0
 >>        aux_cmd_tablep = (struct command **) 0xc03aeb18
 >>        cmd = (struct command *) 0xc0330728
 >>        t = 0
 >>        modif = "\000\032q?h\f2?\200%\000\000?\003\000\000\000\eq?? 
 >> \032q?h\f2??\003\000\000?\032q?\f2??\003\000\000?\003\000\000\r\000 
 >> \000\000?\0172?@?h\000\000\eq??\003\000\000\020\eq??\\\024??\f3?$\\ 
 >> \024?h?>?\200\025;??W\024?\200\025;?@\017;?x\000\000\000\200\215}? 
 >> \000\034q?4\eq?"
 >>        addr = -1070466719
 >>        count = -1
 >>        have_addr = 0
 >>        result = 0
 >> #5  0xc014479e in db_command_loop () at /usr/src/sys/ddb/ 
 >> db_command.c:457
 >> No locals.
 >> #6  0xc01468db in db_trap (type=3, code=0) at /usr/src/sys/ddb/ 
 >> db_trap.c:71
 >>        bkpt = 0
 >> #7  0xc02fdbae in kdb_trap (type=3, code=0, regs=0xe3711bb8) at / 
 >> usr/src/sys/i386/i386/db_interface.c:158
 >>        ddb_mode = 1
 >> #8  0xc030db40 in trap (frame={tf_fs = -479133680, tf_es =  
 >> -479133680, tf_ds = -1070792688, tf_edi = -811750548,
 >>      tf_esi = -1022605312, tf_ebp = -479126508, tf_isp =  
 >> -479126556, tf_ebx = -811750588, tf_edx = 1017, tf_ecx = 3,
 >>      tf_eax = 0, tf_trapno = 3, tf_err = 0, tf_eip = -1070466719,  
 >> tf_cs = 8, tf_eflags = 70, tf_esp = -811750588,
 >>      tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:592
 >>        p = (struct proc *) 0xe37d8d80
 >>        sticks = 13843839841289182128
 >>        i = 0
 >>        ucode = 0
 >>        type = 3
 >>        code = 0
 >>        eva = 0
 >> #9  0xc031f961 in siointr1 (com=0xc30c4800) at machine/cpufunc.h:67
 >>        com = (struct com_s *) 0xc30c4800
 >>        line_status = 249 '?'
 >>        modem_status = 63 '?'
 >>        ioptr = (u_char *) 0x68c040 <Address 0x68c040 out of bounds>
 >>        recv_data = 0 '\000'
 >>        int_ctl = 0 '\000'
 >>        int_ctl_new = 0 '\000'
 >>        tc = (struct timecounter *) 0x68c040
 >>        count = 0
 >> #10 0xc031f89b in siointr (arg=0xc30c4800) at /usr/src/sys/isa/ 
 >> sio.c:1947
 >> No locals.
 >> #11 0xc02fee56 in Xfastintr4 ()
 >> No symbol table info available.
 >> #12 0xc02d16a9 in ufs_vnoperate (ap=0xe3711c88) at /usr/src/sys/ 
 >> ufs/ufs/ufs_vnops.c:2376
 >>        ap = (struct vop_generic_args *) 0x0
 >> #13 0xc01e7b9d in vtruncbuf (vp=0xe195df80, cred=0xc46e5900,  
 >> p=0xe37d8d80, length=196581, blksize=16384)
 >>    at vnode_if.h:1193
 >>        a = {a_desc = 0xc0382880, a_vp = 0xe195df80, a_bp =  
 >> 0xcf9dab44}
 >>        vp = (struct vnode *) 0x0
 >>        bp = (struct buf *) 0xcf9dab44
 >>        blksize = 0
 >>        bp = (struct buf *) 0xcf9dab44
 >>        nbp = (struct buf *) 0xcf9dab6c
 >>        s = 0
 >>        anyfreed = 0
 >>        trunclbn = 12
 >> #14 0xc02c139f in ffs_truncate (vp=0xe195df80, length=196581,  
 >> flags=0, cred=0xc46e5900, p=0xe37d8d80)
 >>    at /usr/src/sys/ufs/ffs/ffs_inode.c:314
 >>        flags = -510271616
 >>        ovp = (struct vnode *) 0xe195df80
 >>        lastblock = 11
 >>        oip = (struct inode *) 0xc4854300
 >>        bn = 0
 >>        lbn = 11
 >>        lastiblock = {-1, -1, -1}
 >>        indir_lbn = {0, 0, -803029019}
 >>        oldblks = {92528, 92536, 92544, 92552, 92560, 92568, 92576,  
 >> 92584, 92592, 92600, 92608, 92616, 0, 0, 0}
 >>        newblks = {92528, 92536, 92544, 92552, 92560, 92568, 92576,  
 >> 92584, 92592, 92600, 92608, 92616, 0, 0, 0}
 >>        fs = (struct fs *) 0xc462c000
 >>        bp = (struct buf *) 0xcf9490c8
 >>        offset = 16357
 >>        size = -510271616
 >>        level = -510271616
 >>        count = 16384
 >>        nblocks = 32
 >>        blocksreleased = 0
 >>        i = 11
 >>        aflags = 1
 >>        error = 0
 >>        allerror = 27
 >>        osize = 196608
 >> #15 0xc02ca002 in ffs_write (ap=0xe3711e64) at /usr/src/sys/ufs/ 
 >> ufs/ufs_readwrite.c:598
 >>        vp = (struct vnode *) 0xe195df80
 >>        uio = (struct uio *) 0xe3711ed4
 >>        ip = (struct inode *) 0xc4854300
 >>        fs = (struct fs *) 0xc462c000
 >>        bp = (struct buf *) 0x0
 >>        p = (struct proc *) 0xe3711ed4
 >>        lbn = -479125804
 >>        osize = 196581
 >>        seqcount = 127
 >>        blkoffset = 0
 >>        error = 27
 >>        extended = 1
 >>        flags = 2130706433
 >>        ioflag = 8323073
 >>        resid = 87
 >>        size = 0
 >>        xfersize = 60
 >>        object = 0xe188b3f4
 >> #16 0xc01ef322 in vn_write (fp=0xc431f540, uio=0xe3711ed4,  
 >> cred=0xc46e5900, flags=0, p=0xe37d8d80) at vnode_if.h:363
 >>        a = {a_desc = 0xc0382100, a_vp = 0xe195df80, a_uio =  
 >> 0xe3711ed4, a_ioflag = 8323073, a_cred = 0xc46e5900}
 >>        vp = (struct vnode *) 0xe195df80
 >>        uio = (struct uio *) 0xe3711ed4
 >>        ioflag = 8323073
 >>        cred = (struct ucred *) 0xc46e5900
 >>        fp = (struct file *) 0xc431f540
 >>        vp = (struct vnode *) 0xe195df80
 >>        error = 8323073
 >>        ioflag = 8323073
 >> #17 0xc01c8899 in dofilewrite (p=0xe37d8d80, fp=0xc431f540, fd=2,  
 >> buf=0x8061e40, nbyte=87, offset=-1, flags=0)
 >>    at /usr/src/sys/sys/file.h:163
 >>        error = -478311040
 >>        fp = (struct file *) 0xc431f540
 >>        cred = (struct ucred *) 0x0
 >>        p = (struct proc *) 0xe37d8d80
 >>        fp = (struct file *) 0xc431f540
 >>        offset = 0
 >>        auio = {uio_iov = 0xe3711eac, uio_iovcnt = 1, uio_offset =  
 >> 196608, uio_resid = 60, uio_segflg = UIO_USERSPACE,
 >>  uio_rw = UIO_WRITE, uio_procp = 0xe37d8d80}
 >>        aiov = {iov_base = 0x8061e5b "ue_enter: create file  
 >> maildrop/381079.82030: File too large\n", iov_len = 60}
 >>        cnt = 87
 >>        error = -478311040
 >>        ktriov = {iov_base = 0x0, iov_len = 0}
 >>        ktruio = {uio_iov = 0xc01bebb0, uio_iovcnt = -479125788,  
 >> uio_offset = 1240175767120, uio_resid = -1070294618,
 >>  uio_segflg = 134656264, uio_rw = UIO_READ, uio_procp = 0x0}
 >>        didktr = 0
 >> #18 0xc01c8752 in write (p=0xe37d8d80, uap=0xe3711f80) at /usr/src/ 
 >> sys/kern/sys_generic.c:329
 >>        p = (struct proc *) 0xe37d8d80
 >>        uap = (struct write_args *) 0xe3711f80
 >>        fp = (struct file *) 0xc431f540
 >>        error = -479125632
 >> #19 0xc030e4a1 in syscall2 (frame={tf_fs = 47, tf_es = 47, tf_ds =  
 >> 47, tf_edi = 2, tf_esi = 0, tf_ebp = -1077937328,
 >>      tf_isp = -479125548, tf_ebx = 134614320, tf_edx = 134614320,  
 >> tf_ecx = 134618688, tf_eax = 4, tf_trapno = 7,
 >>      tf_err = 2, tf_eip = 672270488, tf_cs = 31, tf_eflags = 659,  
 >> tf_esp = -1077937372, tf_ss = 47})
 >>    at /usr/src/sys/i386/i386/trap.c:1175
 >>        params = 0xbfbffb28 "\002"
 >>        i = 0
 >>        callp = (struct sysent *) 0xc038f9a0
 >>        p = (struct proc *) 0xe37d8d80
 >>        orig_tf_eflags = 659
 >>        sticks = 34
 >>        error = 0
 >>        narg = 3
 >>        args = {2, 134618688, 87, 0, 0, 0, 0, 0}
 >>        have_mplock = 1
 >>        code = 4
 >> #20 0xc02fea85 in Xint0x80_syscall ()
 >> No symbol table info available.
 >> #21 0x8054550 in ?? ()
 >> No symbol table info available.
 >> #22 0x8054e80 in ?? ()
 >> No symbol table info available.
 >> #23 0x805292e in ?? ()
 >> No symbol table info available.
 >> #24 0x805277f in ?? ()
 >> No symbol table info available.
 >> #25 0x80526e3 in ?? ()
 >> No symbol table info available.
 >> #26 0x80524bb in ?? ()
 >> No symbol table info available.
 >> #27 0x804d3ba in ?? ()
 >> No symbol table info available.
 >> #28 0x804b07a in ?? ()
 >> No symbol table info available.
 >> #29 0x8049de2 in ?? ()
 >> No symbol table info available.
 >> #30 0x80499ea in ?? ()
 >> No symbol table info available.
 >>
 >>
 >>
 >>
 >>> How-To-Repeat:
 >>>
 >> not known
 >>
 >>> Fix:
 >>>
 >> not known
 >>
 >>> Release-Note:
 >>> Audit-Trail:
 >>> Unformatted:
 >>>
 >> _______________________________________________
 >> freebsd-bugs@freebsd.org mailing list
 >> http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
 >> To unsubscribe, send any mail to "freebsd-bugs- 
 >> unsubscribe@freebsd.org"
 >>
 >>
 >>
 >
 > ----
 > Marc G. Fournier           Hub.Org Networking Services (http:// 
 > www.hub.org)
 > Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ:  
 > 7615664
 >
 



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200508011700.j71H0X6s064092>