From owner-freebsd-stable@FreeBSD.ORG Sat Oct 20 03:20:38 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A250516A419 for ; Sat, 20 Oct 2007 03:20:38 +0000 (UTC) (envelope-from josh.carroll@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.179]) by mx1.freebsd.org (Postfix) with ESMTP id 3BCE313C459 for ; Sat, 20 Oct 2007 03:20:37 +0000 (UTC) (envelope-from josh.carroll@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so1399423pyb for ; Fri, 19 Oct 2007 20:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=/Hy93UUvPFBv3xuA8In47fEFGJmmzHDoMPO8CkwH1x4=; b=a6e9TRXqe5RmDJciPYzwupr8Fu+ND6FlTsBKt/YGfdNEvp+3olRY+gwY50+1RXTUTpAfCH88NQLWHHd8LPEMrd4x2q2LlrDdcUD0T6RQsrX9kw8lfkbL2fGN4p4UodWNZeqGqs5zaO6lke817Y7ALzhwaybprP7EUCEmBa4LHUk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=do5h8SMCtZ0rz9t1E/cwjaoGKFGeUhFgj+Ms/GfoVKmm7yLqMpQXtdVtsUJ1G5yFFy+iwFtWHkAqu5XcFIoZ1BNuQu3avur6Kku+28kjIHl0ibOKB2Rz/62G2XTP5IwVIXmMrieITXhbbMCwttMzviJChbuLmX4/qy4lNiqsa2g= Received: by 10.35.121.12 with SMTP id y12mr2873016pym.1192850436667; Fri, 19 Oct 2007 20:20:36 -0700 (PDT) Received: by 10.35.117.12 with HTTP; Fri, 19 Oct 2007 20:20:36 -0700 (PDT) Message-ID: <8cb6106e0710192020r1b3f3948p41b1b88018a146c6@mail.gmail.com> Date: Fri, 19 Oct 2007 23:20:36 -0400 From: "Josh Carroll" To: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: em lockups during heavy network I/O on RELENG_7 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: josh.carroll@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2007 03:20:38 -0000 Hello, I have managed to lock my (amd64, RELENG_7) machine up twice today. In both cases, I was transferring a file to my laptop (in one case over SMB, the other over FTP). Both resulted in a hard lock (no panic). One of the lockups had an "em1: watchdog timeout" message on the console, the other had no abnormal messages on the console. Both transfers were over 100baseTx-FD ethernet over my em card (em1). This is a new occurrence on RELENG_7. The box was stable on 6.2-RELEASE. The hardware is as follows: Asus P5B motherboard (BIOS revison 1405) Intel Core 2 Quad Q6600 Two Intel em cards (em0: internet facing, em1: LAN facing) Further below is my full kernel config, but here are the options I have that GENERIC does not (not necessarily in order, as they were sorted when I was diff'ing with GENERIC): device atapicam device coretemp device if_bridge device pf device pflog device tap ident PFLOG64 machine amd64 makeoptions COPTFLAGS="-O2 -pipe" options ALTQ options ALTQ_CBQ options ALTQ_HFSC options ALTQ_NOPCC options ALTQ_PRIQ options ALTQ_RED options ALTQ_RIO options COMPAT_LINUX32 options HZ=1000 options LIBICONV options LIBMCHAIN options NETSMB options SC_PIXEL_MODE options SMBFS options UDF options VGA_WIDTH90 I plan on removing the HZ, LIBICONV, LIBMCHAIN, SC_PIXEL_MODE, and VGA_WIDTH90 options and seeing if that helps. Do any of the other options look like likely culprits? If the above doesn't help, is there a way to debug hard lockups? I have 15mbit FiOS and have maxed em0 out without causing this. So I'm guessing either I'm not pushing enough packets through em0 to induce this, or the problem is related to em1 sharing an interrupt with a USB controller: interrupt total rate irq1: atkbd0 6 0 irq6: fdc0 10 0 irq17: uhci1+ 91 0 irq19: uhci3+ 924492 32 irq22: em0 85919 3 irq23: em1 uhci2+ 6627 0 cpu0: timer 56950868 1999 cpu1: timer 56950834 1999 cpu2: timer 56950835 1999 cpu3: timer 56950834 1999 Total 228820516 8035 Anyway, here is the full kernel configuration: makeoptions COPTFLAGS="-O2 -pipe" machine amd64 cpu HAMMER ident PFLOG64 options COMPAT_IA32 options COMPAT_LINUX32 ########################################################################## options SMP # Symmetric MultiProcessor Kernel options STOP_NMI options SCHED_4BSD options PREEMPTION options HZ=1000 options INET # InterNETworking options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server options NETSMB options SMBFS options LIBICONV options LIBMCHAIN options CD9660 # ISO 9660 Filesystem options UDF options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_LABEL # Provides labelization options GEOM_PART_GPT # GUID Partition Tables. options COMPAT_43TTY # Compatible with BSD 4.3 [KEEP THIS!] options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. options ADAPTIVE_GIANT # Giant mutex is adaptive. options AUDIT # Security event auditing # Bus support. Do not remove isa, even if you have no isa slots device pci device acpi # Floppy drives device fdc # coretemp: on-die sensor on Intel Core and newer CPUs device coretemp # ATA and ATAPI devices device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives options ATA_STATIC_ID # Static device numbering # SCSI peripherals device scbus # SCSI bus (required for SCSI) device da # Direct Access (disks) device cd # CD device pass # Passthrough device (direct SCSI access) device atapicam # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device vga # VGA video card driver options VGA_WIDTH90 options SC_PIXEL_MODE device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc device if_bridge # PCI Ethernet NICs. device em # Intel PRO/1000 adapter Gigabit Ethernet Card # Pseudo devices. device loop # Network loopback device random # Entropy device device ether # Ethernet support device tap device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter device pf device pflog # USB support device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device ehci # EHCI PCI->USB interface (USB 2.0) device usb # USB Bus (required) device ugen # Generic device umass # Disks/Mass storage - Requires scbus and da # FireWire support device firewire # FireWire bus code device sbp # SCSI over FireWire (Requires scbus and da) # altq support options ALTQ options ALTQ_CBQ # Class Bases Queuing (CBQ) options ALTQ_RED # Random Early Detection (RED) options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) options ALTQ_PRIQ # Priority Queuing (PRIQ) options ALTQ_NOPCC # Required for SMP build Thanks! Josh