Date: Sat, 29 Nov 2003 18:28:00 -0500 From: Don Bowman <don@sandvine.com> To: 'Kris Kennaway' <kris@obsecurity.org>, stable@FreeBSD.org Cc: phk@freeBSD.org Subject: RE: Corrected gettimeofday() test code Message-ID: <FE045D4D9F7AED4CBFF1B3B813C85337035E3F6B@mail.sandvine.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] From: Kris Kennaway [mailto:kris@obsecurity.org] > > I forwarded the reports of timecounter problems to phk, and he asked > that people who are seeing timecounter problems provide FULL details > of their system configuration, including: > > * dmesg > > * kernel configuration > > * compiler options > > * time-related system configuration (whether ntpd/timed/ntpdate is > running, and if so whether it's correcting for a seriously drifting > clock) > > * The kernel timecounter configuration, e.g. the > kern.timecounter.method and kern.timecounter.hardware sysctls, and > whether changing them has any effect. > > * The exact output of the corrected test program below (the original > would give spurious errors if it didn't run at least once a second, > which may have been confusing some people if their systems were > sufficiently loaded). > > * The system status when the problem is observed (i.e. does it only > occur under load; what else is running at the time) > For this config (below), kern.timecounter.method=0 reproduces the problem, kern.timecounter.method=1 does not. Output in 'error' case: 1070147643.248866 1070147651.028646 1070147643.248866 1070147651.028646 1070147656.287818 1070147664.067692 1070147656.287818 1070147664.067692 1070147659.326429 1070147667.106238 1070147659.326429 1070147667.106238 1070147668.370071 1070147676.149884 1070147668.370071 1070147676.149884 1070147681.433111 1070147689.212926 1070147681.433111 1070147689.212926 1070147683.418743 1070147691.198632 1070147683.418743 1070147691.198632 problem shows up within ~30s of starting the test program, and the messages will come out about once per 1-5s period after that, not regularly. kern.timecounter.hardware: TSC on this machine, i have others which are i8254 which do it too. hw.ncpu=1 compiler flags: COPTFLAGS= -O2 -pipe -malign-loops=4 -malign-jumps=4 -malign-functions=4 -mcpu=i686 -march=i686 -fno-gcse -g machine is running 4.7-RELEASE-p2. dmesg, kernel config attached. Intel-specific functions: Version 00000673: Type 0 - Original OEM Family 6 - Pentium Pro Model 7 - Pentium III/Pentium III Xeon - external L2 cache Stepping 3 Reserved 0 from cpuid. ntpd -p /var/run/ntpd.pid -b -g -A runs. Clock does not drift much, less than 2s/day. Clock is stepped when system boots. There is a chimer on our router which broadcasts ntp time every minute. ntpd is not observed to step clock. On the 8254 machine [a dual 0f27 xeon, 533 FSB with HTT enabled] 1070147645.119531 1070148340.497729 1070148339.-693880469 1070148340.497729 both systems were unloaded. This was with the corrected program you included. --don [-- Attachment #2 --] Copyright (c) 1992-2002 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.7-RELEASE-p2 #1: 20021212_13:58:28 Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 551252183 Hz CPU: Pentium III/Pentium III Xeon/Celeron (551.25-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x673 Stepping = 3 Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE> real memory = 536854528 (524272K bytes) avail memory = 518111232 (505968K bytes) Preloaded elf kernel "kernel" at 0xc042a000. Preloaded elf module "if_xl.ko" at 0xc042a09c. Preloaded elf module "miibus.ko" at 0xc042a13c. netsmb_dev: loaded Pentium Pro MTRR support enabled md0: Malloc disk Using $PIR table, 8 entries at 0xc00f0e70 npx0: <math processor> on motherboard npx0: INT 16 interface pcib0: <Intel 82443BX (440 BX) host to PCI bridge> on motherboard pci0: <PCI bus> on pcib0 pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> at device 1.0 on pci0 pci1: <PCI bus> on pcib1 pci1: <ATI Rage128-RF graphics accelerator> at 0.0 irq 11 isab0: <Intel 82371AB PCI to ISA bridge> at device 4.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel PIIX4 ATA33 controller> port 0xb800-0xb80f at device 4.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 uhci0: <Intel 82371AB/EB (PIIX4) USB controller> port 0xb400-0xb41f at device 4.2 on pci0 pci_cfgintr_virgin: using routable interrupt 3 pci_cfgintr: 0:4 INTD routed to irq 3 usb0: <Intel 82371AB/EB (PIIX4) USB controller> on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered intpm0: <Intel 82371AB Power management controller> port 0xe800-0xe80f irq 9 at device 4.3 on pci0 intpm0: I/O mapped e800 intpm0: intr IRQ 9 enabled revision 0 smbus0: <System Management Bus> on intsmb0 smb0: <SMBus general purpose I/O> on smbus0 intpm0: PM I/O mapped e400 xl0: <3Com 3c905B-TX Fast Etherlink XL> port 0xb000-0xb07f mem 0xde800000-0xde80007f irq 10 at device 11.0 on pci0 xl0: Ethernet address: 00:01:02:ed:38:00 miibus0: <MII bus> on xl0 xlphy0: <3Com internal media interface> on miibus0 xlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto xl1: <3Com 3c905B-TX Fast Etherlink XL> port 0xa800-0xa87f mem 0xde000000-0xde00007f irq 11 at device 12.0 on pci0 xl1: Ethernet address: 00:50:04:01:2f:8e miibus1: <MII bus> on xl1 xlphy1: <3Com internal media interface> on miibus1 xlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto orm0: <Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc87ff on isa0 fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1: configured irq 3 not in bitmap of probed irqs 0 sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A IP packet filtering initialized, divert enabled, rule-based forwarding enabled, default to accept, logging disabled ad0: 19470MB <IBM-DJNA-352030> [39560/16/63] at ata0-master UDMA33 acd0: CDROM <MATSHITA CR-589> at ata0-slave PIO4 Mounting root from ufs:/dev/ad0s1a [-- Attachment #3 --] ___# ___# GENERIC -- Generic kernel configuration file for FreeBSD/i386 ___# ___# For more information on this file, please read the handbook section on ___# Kernel Configuration Files: ___# ___# http://www.FreeBSD.org/handbook/kernelconfig-config.html ___# ___# The handbook is also available locally in /usr/share/doc/handbook ___# if you've installed the doc distribution, otherwise always see the ___# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the ___# latest information. ___# ___# An exhaustive list of options and more detailed explanations of the ___# device lines is also present in the ./LINT configuration file. If you are ___# in doubt as to the purpose or necessity of a line, check first in LINT. ___# ___# $FreeBSD: src/sys/i386/conf/GENERIC,v 1.246.2.39 2002/03/24 13:19:10 wilko Exp $ ___machine i386 ___#cpu I386_CPU ___#cpu I486_CPU ___cpu I586_CPU ___cpu I686_CPU ___ident TPC ___maxusers 0 ___makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols ___#options MATH_EMULATE #Support for x87 emulation ___options INET #InterNETworking ___#options INET6 #IPv6 communications protocols ___options FFS #Berkeley Fast Filesystem ___options FFS_ROOT #FFS usable as root device [keep this!] ___options SOFTUPDATES #Enable FFS soft updates support ___options UFS_DIRHASH #Improve performance on big directories ___options MFS #Memory Filesystem ___options MD_ROOT #MD is a potential root device ___options NFS #Network Filesystem ___options NFS_ROOT #NFS usable as root device, NFS required ___options MSDOSFS #MSDOS Filesystem ___options CD9660 #ISO 9660 Filesystem ___options CD9660_ROOT #CD-ROM usable as root, CD9660 required ___options PROCFS #Process filesystem ___options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] ___options SCSI_DELAY=2000 #Delay (in ms) before probing SCSI ___options UCONSOLE #Allow users to grab the console ___options USERCONFIG #boot -c editor ___options VISUAL_USERCONFIG #visual boot -c editor ___options KTRACE #ktrace(1) support ___options SYSVSHM #SYSV-style shared memory ___options SYSVMSG #SYSV-style message queues ___options SYSVSEM #SYSV-style semaphores ___options P1003_1B #Posix P1003_1B real-time extensions ___options _KPOSIX_PRIORITY_SCHEDULING ___options ICMP_BANDLIM #Rate limit bad replies ___options KBD_INSTALL_CDEV # install a CDEV entry in /dev ___# To make an SMP kernel, the next two are needed ___#options SMP # Symmetric MultiProcessor Kernel ___#options APIC_IO # Symmetric (APIC) I/O ___device isa ___device eisa ___device pci ___# Floppy drives ___device fdc0 at isa? port IO_FD1 irq 6 drq 2 ___device fd0 at fdc0 drive 0 ___device fd1 at fdc0 drive 1 ___# ___# ATA and ATAPI devices ___device ata0 at isa? port IO_WD1 irq 14 ___device ata1 at isa? port IO_WD2 irq 15 ___device ata ___device atadisk # ATA disk drives ___device atapicd # ATAPI CDROM drives ___device atapifd # ATAPI floppy drives ___device atapist # ATAPI tape drives ___options ATA_STATIC_ID #Static device numbering ___# SCSI Controllers ___device ahb # EISA AHA1742 family ___device ahc # AHA2940 and onboard AIC7xxx devices ___device amd # AMD 53C974 (Tekram DC-390(T)) ___device isp # Qlogic family ___device ncr # NCR/Symbios Logic ___device sym # NCR/Symbios Logic (newer chipsets) ___options SYM_SETUP_LP_PROBE_MAP=0x40 ___ # Allow ncr to attach legacy NCR devices when ___ # both sym and ncr are configured ___device adw ___device ncv # NCR 53C500 ___device nsp # Workbit Ninja SCSI-3 ___device stg # TMC 18C30/18C50 ___# SCSI peripherals ___device scbus # SCSI bus (required) ___device da # Direct Access (disks) ___device sa # Sequential Access (tape etc) ___device cd # CD ___device pass # Passthrough device (direct SCSI access) ___# RAID controllers interfaced to the SCSI subsystem ___device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID ___device dpt # DPT Smartcache - See LINT for options! ___device mly # Mylex AcceleRAID/eXtremeRAID ___device ciss # Compaq SmartRAID 5* series ___# RAID controllers ___device aac # Adaptec FSA RAID, Dell PERC2/PERC3 ___device ida # Compaq Smart RAID ___device amr # AMI MegaRAID ___device mlx # Mylex DAC960 family ___device twe # 3ware Escalade ___# atkbdc0 controls both the keyboard and the PS/2 mouse ___device atkbdc0 at isa? port IO_KBD ___device atkbd0 at atkbdc? irq 1 flags 0x1 ___device psm0 at atkbdc? irq 12 ___device vga0 at isa? ___# splash screen/screen saver ___pseudo-device splash ___# syscons is the default console driver, resembling an SCO console ___device sc0 at isa? flags 0x100 ___# Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver ___#device vt0 at isa? ___#options XSERVER # support for X server on a vt console ___#options FAT_CURSOR # start with block cursor ___# If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines ___#options PCVT_SCANSET=2 # IBM keyboards are non-std ___# Floating point support - do not disable. ___device npx0 at nexus? port IO_NPX irq 13 ___# Power management support (see LINT for more options) ___#device apm0 at nexus? disable flags 0x20 # Advanced Power Management ___# Serial (COM) ports ___device sio0 at isa? port IO_COM1 flags 0x10 irq 4 ___device sio1 at isa? port IO_COM2 irq 3 ___# PCI Ethernet NICs that use the common MII bus controller code. ___# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! ___#device miibus # MII bus support ___#device fxp # Intel EtherExpress PRO/100B (82557, 82558) ___#device wx # Intel Gigabit Ethernet Card (``Wiseman'') ___#device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') ___#device bge # Broadcom BCM570x (``Tigon III'') ___#device gx ___#device em # Intel Gigabit cards ___# Pseudo devices - the number indicates how many units to allocate. ___pseudo-device loop # Network loopback ___pseudo-device ether # Ethernet support ___#pseudo-device sl 1 # Kernel SLIP ___#pseudo-device ppp 1 # Kernel PPP ___pseudo-device tun # Packet tunnel. ___pseudo-device pty # Pseudo-ttys (telnet etc) ___pseudo-device md # Memory "disks" ___#pseudo-device gif # IPv6 and IPv4 tunneling ___#pseudo-device faith 1 # IPv6-to-IPv4 relaying (translation) ___# The `bpf' pseudo-device enables the Berkeley Packet Filter. ___# Be aware of the administrative consequences of enabling this! ___pseudo-device bpf #Berkeley packet filter ___# USB support ___device uhci # UHCI PCI->USB interface ___device ohci # OHCI PCI->USB interface ___device usb # USB Bus (required) ___device ugen # Generic ___device uhid # "Human Interface Devices" ___device ukbd # Keyboard ___device ulpt # Printer ___device umass # Disks/Mass storage - Requires scbus and da ___device ums # Mouse ___device uscanner # Scanners ___device urio # Diamond Rio MP3 Player ___# USB Ethernet, requires mii ___#device aue # ADMtek USB ethernet ___#device cue # CATC USB ethernet ___#device kue # Kawasaki LSI USB ethernet ___# Local options ___options HZ=1000 ___options INCLUDE_CONFIG_FILE # Include this file in kernel ___options IPFIREWALL #firewall ___options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default ___options IPFIREWALL_FORWARD #enable transparent proxy support ___options IPDIVERT #divert sockets ___#options DUMMYNET ___#options BRIDGE ___options RANDOM_IP_ID ___#options TCP_DROP_SYNFIN #drop TCP packets with SYN+FIN ___options EXT2FS #Add linux FS support ___options MAXDSIZ="(512*1024*1024)" ___options DFLDSIZ="(512*1024*1024)" ___options NETSMB #SMB/CIFS requester ___options NETSMBCRYPTO #encrypted password support for SMB ___options LIBMCHAIN #mbuf management library ___options LIBICONV #Kernel side iconv library ___options SMBFS #SMB/CIFS filesystem ___options CONSPEED=115200 # speed for serial console ___options USER_LDT #allow user-level control of i386 ldt ___options CPU_ENABLE_SSE ___#options IPFW2 ___device smbus ___device iicbus ___device iicbb ___device intpm ___device alpm ___device ichsmb ___device smb ___device pcf
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?FE045D4D9F7AED4CBFF1B3B813C85337035E3F6B>