Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 17 Feb 2009 22:41:00 GMT
From:      Colin Faber <cfaber@gmail.com>
To:        freebsd-gnats-submit@FreeBSD.org
Subject:   kern/131796: Introducing new USB or eSATA disks results in atapci1+ irq storm until reboot
Message-ID:  <200902172241.n1HMf0MW089058@www.freebsd.org>
Resent-Message-ID: <200902172250.n1HMo39b022312@freefall.freebsd.org>

next in thread | raw e-mail | index | archive | help

>Number:         131796
>Category:       kern
>Synopsis:       Introducing new USB or eSATA disks results in atapci1+ irq storm until reboot
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Feb 17 22:50:03 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Colin Faber
>Release:        7.1-RELENG-p2 amd64 kernel
>Organization:
n/a
>Environment:
System: FreeBSD hal.fpsn 7.1-RELEASE-p2 FreeBSD 7.1-RELEASE-p2 #2: Sun Feb 15 12:21:19 MST 2009 cfaber@hal.fpsn:/usr/obj/usr/src/sys/HAL amd64

>Description:
It appears that when plugging in any USB, or eSATA disk drive, atapci1+ gets stuck in a massive irq storm, running about 13k irqs a second. This results in about 90% interrupt load on the system and makes everything unstable and slow.

Note - this load appears even with ZERO activity on the device in question and little or no geom activity.

Also Note - this machine is running two geom_raid5 array's made of geom labeled providers


Non-storm:
cfaber@hal:~$ vmstat -i
interrupt                          total       rate
irq1: atkbd0                           6          0
irq14: ata0                         4316          0
irq16: mpt0                      1782986        243
irq21: ohci0                      213266         29
irq23: atapci1+                  2182108        298
cpu0: timer                     14628721       1999
irq256: nfe0                      215036         29
Total                           19026439       2601


Storm:
cfaber@hal:~$ vmstat -i
interrupt                          total       rate
irq1: atkbd0                           6          0
irq14: ata0                         1534          1
irq16: mpt0                          674          0
irq21: ohci0                        1169          0
irq22: ehci0                         193          0
irq23: atapci1+                 20420307      14230
cpu0: timer                      2868072       1998
irq256: nfe0                        1171          0
Total                           23293126      16232

>How-To-Repeat:
cvsup to RELENG_7_1

configure an amd64 kernel with the following parameters:

cpu		HAMMER
ident		HAL
options 	GEOM_JOURNAL
options 	SCHED_ULE		# ULE scheduler
options 	PREEMPTION		# Enable kernel thread preemption
options 	INET			# InterNETworking
options 	FFS			# Berkeley Fast Filesystem
options 	SOFTUPDATES		# Enable FFS soft updates support
options 	UFS_ACL			# Support for access control lists
options 	UFS_DIRHASH		# Improve performance on big directories
options 	UFS_GJOURNAL		# Enable gjournal-based UFS journaling
options 	MD_ROOT			# MD is a potential root device
options 	NFSCLIENT		# Network Filesystem Client
options 	NFSSERVER		# Network Filesystem Server
options 	NFSLOCKD		# Network Lock Manager
options 	NTFS			# NT File System
options 	MSDOSFS			# MSDOS Filesystem
options 	CD9660			# ISO 9660 Filesystem
options 	PROCFS			# Process filesystem (requires PSEUDOFS)
options 	PSEUDOFS		# Pseudo-filesystem framework
options 	GEOM_PART_GPT		# GUID Partition Tables.
options 	GEOM_LABEL		# Provides labelization
options 	COMPAT_43TTY		# BSD 4.3 TTY compat [KEEP THIS!]
options 	COMPAT_IA32		# Compatible with i386 binaries
options 	COMPAT_FREEBSD4		# Compatible with FreeBSD4
options 	COMPAT_FREEBSD5		# Compatible with FreeBSD5
options 	COMPAT_FREEBSD6		# Compatible with FreeBSD6
options 	SCSI_DELAY=5000		# Delay (in ms) before probing SCSI
options 	KTRACE			# ktrace(1) support
options 	STACK			# stack(9) support
options 	SYSVSHM			# SYSV-style shared memory
options 	SYSVMSG			# SYSV-style message queues
options 	SYSVSEM			# SYSV-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
options 	ADAPTIVE_GIANT		# Giant mutex is adaptive.
options 	STOP_NMI		# Stop CPUS using NMI instead of IPI
options 	AUDIT			# Security event auditing
options         COMPAT_LINUX32
options         LINPROCFS
options 	SMP			# Symmetric MultiProcessor Kernel
device		cpufreq
device		acpi
device		pci
device		ata
device		atadisk		# ATA disk drives
device		ataraid		# ATA RAID drives
device		atapicd		# ATAPI CDROM drives
device		atapifd		# ATAPI floppy drives
device		atapist		# ATAPI tape drives
options 	ATA_STATIC_ID	# Static device numbering
device		mpt		# LSI-Logic MPT-Fusion
device		scbus		# SCSI bus (required for SCSI)
device		ch		# SCSI media changers
device		da		# Direct Access (disks)
device		sa		# Sequential Access (tape etc)
device		cd		# CD
device		pass		# Passthrough device (direct SCSI access)
device		ses		# SCSI Environmental Services (and SAF-TE)
device		atkbdc		# AT keyboard controller
device		atkbd		# AT keyboard
device		psm		# PS/2 mouse
device		kbdmux		# keyboard multiplexer
device		vga		# VGA video card driver
device		splash		# Splash screen and screen saver support
device		sc
device		miibus		# MII bus support
device		nfe		# nVidia nForce MCP on-board Ethernet
device		loop		# Network loopback
device		random		# Entropy device
device		ether		# Ethernet support
device		pty		# Pseudo-ttys (telnet etc)
device		md		# Memory "disks"
device		firmware	# firmware assist module
device		bpf		# Berkeley packet filter
device		uhci		# UHCI PCI->USB interface
device		ohci		# OHCI PCI->USB interface
device		ehci		# EHCI PCI->USB interface (USB 2.0)
device		usb		# USB Bus (required)
device		ugen		# Generic
device		uhid		# "Human Interface Devices"
device		ukbd		# Keyboard
device		ulpt		# Printer
device		umass		# Disks/Mass storage - Requires scbus and da
device		ums		# Mouse
device		uscanner	# Scanners
device		ucom		# Generic com ttys
device		sound		# Provide general sound support


Next, connect any umass device, or sata device. atapci1+ will go nuts the second the kernel sees the device online.
>Fix:
So far, the only way to get the problem to stop is to reboot the system. If the device is connected already, then there are no problems with irq storms on atapci1+.

I'm willing to try and debugging procedures provided. As well as provide additional information once requested.

>Release-Note:
>Audit-Trail:
>Unformatted:



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200902172241.n1HMf0MW089058>