From owner-freebsd-net@freebsd.org Mon Oct 19 14:20:26 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83B40A18F7A for ; Mon, 19 Oct 2015 14:20:26 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 606931865 for ; Mon, 19 Oct 2015 14:20:26 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 5D2F6A18F78; Mon, 19 Oct 2015 14:20:26 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CB84A18F77 for ; Mon, 19 Oct 2015 14:20:26 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D35F51864; Mon, 19 Oct 2015 14:20:25 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by lbbwb3 with SMTP id wb3so83911206lbb.1; Mon, 19 Oct 2015 07:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=E9/V31xGJa61hKG2qfyAqbFTMw63SDMZNygjzFSLYlA=; b=dCGL0Xmwntm5kHVHFlolnkuqdirrdh8k60sIvZJZmBbQ7V2/dYnimnJQoHeXoW40dD 3F3fcQ/ruPyEHhBco/s0Xrp9IwZFbhzOdYQxvgRXQyaZdkzFd2IgTeLpsmFNDY1Ya+/X W2vsvqPrtTZYyNmuDi1NiZdSPCC3PNAjweDCKUzlWazRVih7cdEQFZLprL1MZkYp4jsW foff9/UdiocCwQCqDw05jYzR26QnkuW0RD+Y5MRrlFnzbqnfHFQPaPLa38KO6jxL2W7c tHsH3Q0cpcyy7EHd8otDIuKLcLNa03lBbo4wmVUfa3fuPnsJhtl/32G3GSOIUgSb6KM2 Cduw== MIME-Version: 1.0 X-Received: by 10.112.167.101 with SMTP id zn5mr14698187lbb.18.1445264423910; Mon, 19 Oct 2015 07:20:23 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.177.4 with HTTP; Mon, 19 Oct 2015 07:20:23 -0700 (PDT) In-Reply-To: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> References: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> Date: Mon, 19 Oct 2015 07:20:23 -0700 X-Google-Sender-Auth: _ac3sdREsU6bohIquX8TSNkYOFA Message-ID: Subject: Re: ixl 40G bad performance? From: Luigi Rizzo To: "Eggert, Lars" Cc: "net@freebsd.org" , "jfv@FreeBSD.org" , "ricera10@gmail.com" , Giuseppe Lettieri Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2015 14:20:26 -0000 i would look at the following: - c states and clock speed - make sure you never go below C1, and fix the clock speed to max. Sure these parameters also affect the 10G card, but there may be strange interaction that trigger the power saving modes in different ways - interrupt moderation (may affect ping latency, do not remember how it is set in ixl but probably a sysctl - number of queues (32 is a lot i wouldn't use more than 4-8), may affect cpu-socket affinity - tso and flow director - i have seen bad effects of accelerations so i would run the iperf test with of these features disabled on both sides, and then enable them one at a time - queue sizes - the driver seems to use 1024 slots which is about 1.5 MB queued, which in turn means you have 300us (and possibly half of that) to drain the queue at 40Gbit/s. 150-300us may seem an eternity, but if a couple of cores fall into c7 your budget is gone and the loss will trigger a retransmission and window halving etc. cheers luigi On Mon, Oct 19, 2015 at 6:52 AM, Eggert, Lars wrote: > Hi, > > I'm running a few simple tests on -CURRENT with a pair of dual-port Intel XL710 boards, which are seen by the kernel as: > > ixl0: mem 0xdc800000-0xdcffffff,0xdd808000-0xdd80ffff irq 32 at device 0.0 on pci3 > ixl0: Using MSIX interrupts with 33 vectors > ixl0: f4.40 a1.4 n04.53 e80001dca > ixl0: Using defaults for TSO: 65518/35/2048 > ixl0: Ethernet address: 68:05:ca:32:0b:98 > ixl0: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl0: netmap queues/slots: TX 32/1024, RX 32/1024 > ixl1: mem 0xdc000000-0xdc7fffff,0xdd800000-0xdd807fff irq 32 at device 0.1 on pci3 > ixl1: Using MSIX interrupts with 33 vectors > ixl1: f4.40 a1.4 n04.53 e80001dca > ixl1: Using defaults for TSO: 65518/35/2048 > ixl1: Ethernet address: 68:05:ca:32:0b:99 > ixl1: PCI Express Bus: Speed 8.0GT/s Width x8 > ixl1: netmap queues/slots: TX 32/1024, RX 32/1024 > ixl0: link state changed to UP > ixl1: link state changed to UP > > I have two identical machines connected with patch cables (no switch). iperf performance is bad: > > # iperf -c 10.0.1.2 > ------------------------------------------------------------ > Client connecting to 10.0.1.2, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 10.0.1.1 port 19238 connected with 10.0.1.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 3.91 GBytes 3.36 Gbits/sec > > As is flood ping latency: > > # sudo ping -f 10.0.1.2 > PING 10.0.1.2 (10.0.1.2): 56 data bytes > .^C > --- 10.0.1.2 ping statistics --- > 41927 packets transmitted, 41926 packets received, 0.0% packet loss > round-trip min/avg/max/stddev = 0.084/0.116/0.145/0.002 ms > > Any ideas on what's going on here? Testing 10G ix interfaces between the same two machines results in 9.39 Gbits/sec and flood ping latencies of 17 usec. > > Thanks, > Lars > > PS: Full dmesg attached. > > Copyright (c) 1992-2015 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 11.0-CURRENT #2 483de3c(muclab)-dirty: Mon Oct 19 11:01:16 CEST 2015 > elars@laurel.muccbc.hq.netapp.com:/usr/home/elars/obj/usr/home/elars/src/sys/MUCLAB amd64 > FreeBSD clang version 3.7.0 (tags/RELEASE_370/final 246257) 20150906 > VT(vga): resolution 640x480 > CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz (2000.05-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 > Features=0xbfebfbff > Features2=0x1fbee3ff > AMD Features=0x2c100800 > AMD Features2=0x1 > XSAVE Features=0x1 > VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID > TSC: P-state invariant, performance statistics > real memory = 137438953472 (131072 MB) > avail memory = 133484290048 (127300 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: < > > FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs > FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 1 > cpu2 (AP): APIC ID: 2 > cpu3 (AP): APIC ID: 3 > cpu4 (AP): APIC ID: 4 > cpu5 (AP): APIC ID: 5 > cpu6 (AP): APIC ID: 6 > cpu7 (AP): APIC ID: 7 > cpu8 (AP): APIC ID: 8 > cpu9 (AP): APIC ID: 9 > cpu10 (AP): APIC ID: 10 > cpu11 (AP): APIC ID: 11 > cpu12 (AP): APIC ID: 12 > cpu13 (AP): APIC ID: 13 > cpu14 (AP): APIC ID: 14 > cpu15 (AP): APIC ID: 15 > cpu16 (AP): APIC ID: 32 > cpu17 (AP): APIC ID: 33 > cpu18 (AP): APIC ID: 34 > cpu19 (AP): APIC ID: 35 > cpu20 (AP): APIC ID: 36 > cpu21 (AP): APIC ID: 37 > cpu22 (AP): APIC ID: 38 > cpu23 (AP): APIC ID: 39 > cpu24 (AP): APIC ID: 40 > cpu25 (AP): APIC ID: 41 > cpu26 (AP): APIC ID: 42 > cpu27 (AP): APIC ID: 43 > cpu28 (AP): APIC ID: 44 > cpu29 (AP): APIC ID: 45 > cpu30 (AP): APIC ID: 46 > cpu31 (AP): APIC ID: 47 > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > ioapic2 irqs 48-71 on motherboard > random: entropy device external interface > module_register_init: MOD_LOAD (vesa, 0xffffffff8094fb90, 0) error 19 > netmap: loaded module > vtvga0: on motherboard > smbios0: at iomem 0xf04d0-0xf04ee on motherboard > smbios0: Version: 2.7, BCD Revision: 2.7 > cryptosoft0: on motherboard > aesni0: on motherboard > acpi0: on motherboard > acpi0: Power Button (fixed) > cpu0: on acpi0 > cpu1: on acpi0 > cpu2: on acpi0 > cpu3: on acpi0 > cpu4: on acpi0 > cpu5: on acpi0 > cpu6: on acpi0 > cpu7: on acpi0 > cpu8: on acpi0 > cpu9: on acpi0 > cpu10: on acpi0 > cpu11: on acpi0 > cpu12: on acpi0 > cpu13: on acpi0 > cpu14: on acpi0 > cpu15: on acpi0 > cpu16: on acpi0 > cpu17: on acpi0 > cpu18: on acpi0 > cpu19: on acpi0 > cpu20: on acpi0 > cpu21: on acpi0 > cpu22: on acpi0 > cpu23: on acpi0 > cpu24: on acpi0 > cpu25: on acpi0 > cpu26: on acpi0 > cpu27: on acpi0 > cpu28: on acpi0 > cpu29: on acpi0 > cpu30: on acpi0 > cpu31: on acpi0 > attimer0: port 0x40-0x43 irq 0 on acpi0 > Timecounter "i8254" frequency 1193182 Hz quality 0 > Event timer "i8254" frequency 1193182 Hz quality 100 > atrtc0: port 0x70-0x71 irq 8 on acpi0 > Event timer "RTC" frequency 32768 Hz quality 0 > hpet0: iomem 0xfed00000-0xfed003ff on acpi0 > Timecounter "HPET" frequency 14318180 Hz quality 950 > Event timer "HPET" frequency 14318180 Hz quality 350 > Event timer "HPET1" frequency 14318180 Hz quality 340 > Event timer "HPET2" frequency 14318180 Hz quality 340 > Event timer "HPET3" frequency 14318180 Hz quality 340 > Event timer "HPET4" frequency 14318180 Hz quality 340 > Event timer "HPET5" frequency 14318180 Hz quality 340 > Event timer "HPET6" frequency 14318180 Hz quality 340 > Event timer "HPET7" frequency 14318180 Hz quality 340 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > pcib1: irq 26 at device 1.0 on pci0 > pci1: on pcib1 > pcib2: irq 26 at device 1.1 on pci0 > pci2: on pcib2 > igb0: port 0x8020-0x803f mem 0xdf820000-0xdf83ffff,0xdf844000-0xdf847fff irq 27 at device 0.0 on pci2 > igb0: Using MSIX interrupts with 9 vectors > igb0: Ethernet address: 00:25:90:9b:73:2e > igb0: Bound queue 0 to cpu 0 > igb0: Bound queue 1 to cpu 1 > igb0: Bound queue 2 to cpu 2 > igb0: Bound queue 3 to cpu 3 > igb0: Bound queue 4 to cpu 4 > igb0: Bound queue 5 to cpu 5 > igb0: Bound queue 6 to cpu 6 > igb0: Bound queue 7 to cpu 7 > igb0: netmap queues/slots: TX 8/1024, RX 8/1024 > igb1: port 0x8000-0x801f mem 0xdf800000-0xdf81ffff,0xdf840000-0xdf843fff irq 30 at device 0.1 on pci2 > igb1: Using MSIX interrupts with 9 vectors > igb1: Ethernet address: 00:25:90:9b:73:2f > igb1: Bound queue 0 to cpu 8 > igb1: Bound queue 1 to cpu 9 > igb1: Bound queue 2 to cpu 10 > igb1: Bound queue 3 to cpu 11 > igb1: Bound queue 4 to cpu 12 > igb1: Bound queue 5 to cpu 13 > igb1: Bound queue 6 to cpu 14 > igb1: Bound queue 7 to cpu 15 > igb1: netmap queues/slots: TX 8/1024, RX 8/1024 > pcib3: irq 33 at device 2.0 on pci0 > pci3: on pcib3 > pci3: at device 0.0 (no driver attached) > pci3: at device 0.1 (no driver attached) > pcib4: irq 33 at device 2.2 on pci0 > pci4: on pcib4 > pcib5: irq 41 at device 3.0 on pci0 > pci5: on pcib5 > pcib6: irq 41 at device 3.2 on pci0 > pci6: on pcib6 > ix0: port 0x7020-0x703f mem 0xdf180000-0xdf1fffff,0xdf604000-0xdf607fff irq 42 at device 0.0 on pci6 > ix0: Using MSIX interrupts with 33 vectors > ix0: Ethernet address: 90:e2:ba:77:d4:9c > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix0: netmap queues/slots: TX 32/2048, RX 32/2048 > ix1: port 0x7000-0x701f mem 0xdf100000-0xdf17ffff,0xdf600000-0xdf603fff irq 45 at device 0.1 on pci6 > ix1: Using MSIX interrupts with 33 vectors > ix1: Ethernet address: 90:e2:ba:77:d4:9d > ix1: PCI Express Bus: Speed 5.0GT/s Width x8 > ix1: netmap queues/slots: TX 32/2048, RX 32/2048 > pcib7: irq 16 at device 17.0 on pci0 > pci7: on pcib7 > isci0: port 0x6000-0x60ff mem 0xde07c000-0xde07ffff,0xddc00000-0xddffffff irq 16 at device 0.0 on pci7 > pci0: at device 22.0 (no driver attached) > pci0: at device 22.1 (no driver attached) > ehci0: mem 0xdf923000-0xdf9233ff irq 16 at device 26.0 on pci0 > usbus0: EHCI version 1.0 > usbus0 on ehci0 > pcib8: irq 17 at device 28.0 on pci0 > pci8: on pcib8 > pcib9: irq 19 at device 28.7 on pci0 > pci9: on pcib9 > pcib10: at device 0.0 on pci9 > pci10: on pcib10 > pcib11: at device 0.0 on pci10 > pci11: on pcib11 > pcib12: at device 0.0 on pci11 > pci12: on pcib12 > vgapci0: mem 0xdb000000-0xdbffffff,0xdf000000-0xdf003fff,0xde800000-0xdeffffff irq 19 at device 0.0 on pci12 > vgapci0: Boot video device > pcib13: at device 1.0 on pci10 > pci13: on pcib13 > ehci1: mem 0xdf922000-0xdf9223ff irq 23 at device 29.0 on pci0 > usbus1: EHCI version 1.0 > usbus1 on ehci1 > pcib14: at device 30.0 on pci0 > pci14: on pcib14 > isab0: at device 31.0 on pci0 > isa0: on isab0 > ahci0: port 0x9050-0x9057,0x9040-0x9043,0x9030-0x9037,0x9020-0x9023,0x9000-0x901f mem 0xdf921000-0xdf9217ff irq 18 at device 31.2 on pci0 > ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported > ahcich0: at channel 0 on ahci0 > ahcich1: at channel 1 on ahci0 > ahcich2: at channel 2 on ahci0 > ahcich3: at channel 3 on ahci0 > ahcich4: at channel 4 on ahci0 > ahcich5: at channel 5 on ahci0 > ahciem0: on ahci0 > ichsmb0: port 0x1180-0x119f mem 0xdf920000-0xdf9200ff irq 18 at device 31.3 on pci0 > smbus0: on ichsmb0 > smb0: on smbus0 > pcib15: on acpi0 > pci15: on pcib15 > pcib16: on acpi0 > pci16: on pcib16 > pcib17: irq 57 at device 2.0 on pci16 > pci17: on pcib17 > pcib18: irq 64 at device 3.0 on pci16 > pci18: on pcib18 > pcib19: irq 64 at device 3.2 on pci16 > pci19: on pcib19 > pcib20: on acpi0 > pci20: on pcib20 > acpi_button0: on acpi0 > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > uart0: console (115200,n,8,1) > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > ipmi0: port 0xca2,0xca3 on acpi0 > ipmi0: KCS mode found at io 0xca2 on acpi > ichwd0 on isa0 > ichwd0: ICH WDT present but disabled in BIOS or hardware > device_attach: ichwd0 attach returned 6 > ichwd0 at port 0x430-0x437,0x460-0x47f on isa0 > ichwd0: ICH WDT present but disabled in BIOS or hardware > device_attach: ichwd0 attach returned 6 > orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0 > coretemp0: on cpu0 > est0: on cpu0 > coretemp1: on cpu1 > est1: on cpu1 > coretemp2: on cpu2 > est2: on cpu2 > coretemp3: on cpu3 > est3: on cpu3 > coretemp4: on cpu4 > est4: on cpu4 > coretemp5: on cpu5 > est5: on cpu5 > coretemp6: on cpu6 > est6: on cpu6 > coretemp7: on cpu7 > est7: on cpu7 > coretemp8: on cpu8 > est8: on cpu8 > coretemp9: on cpu9 > est9: on cpu9 > coretemp10: on cpu10 > est10: on cpu10 > coretemp11: on cpu11 > est11: on cpu11 > coretemp12: on cpu12 > est12: on cpu12 > coretemp13: on cpu13 > est13: on cpu13 > coretemp14: on cpu14 > est14: on cpu14 > coretemp15: on cpu15 > est15: on cpu15 > coretemp16: on cpu16 > est16: on cpu16 > coretemp17: on cpu17 > est17: on cpu17 > coretemp18: on cpu18 > est18: on cpu18 > coretemp19: on cpu19 > est19: on cpu19 > coretemp20: on cpu20 > est20: on cpu20 > coretemp21: on cpu21 > est21: on cpu21 > coretemp22: on cpu22 > est22: on cpu22 > coretemp23: on cpu23 > est23: on cpu23 > coretemp24: on cpu24 > est24: on cpu24 > coretemp25: on cpu25 > est25: on cpu25 > coretemp26: on cpu26 > est26: on cpu26 > coretemp27: on cpu27 > est27: on cpu27 > coretemp28: on cpu28 > est28: on cpu28 > coretemp29: on cpu29 > est29: on cpu29 > coretemp30: on cpu30 > est30: on cpu30 > coretemp31: on cpu31 > est31: on cpu31 > fuse-freebsd: version 0.4.4, FUSE ABI 7.8 > Timecounters tick every 1.000 msec > iw_cxgb: Chelsio T3 RDMA Driver loaded > IPsec: Initialized Security Association Processing. > ipfw2 (+ipv6) initialized, divert loadable, nat enabled, default to accept, logging disabled > DUMMYNET 0 with IPv6 initialized (100409) > load_dn_sched dn_sched FIFO loaded > load_dn_sched dn_sched PRIO loaded > load_dn_sched dn_sched QFQ loaded > load_dn_sched dn_sched RR loaded > load_dn_sched dn_sched WF2Q+ loaded > ipmi0: IPMI device rev. 1, firmware rev. 2.35, version 2.0 > ipmi0: Number of channels 3 > ipmi0: Attached watchdog > usbus0: 480Mbps High Speed USB v2.0 > usbus1: 480Mbps High Speed USB v2.0 > ugen0.1: at usbus0 > uhub0: on usbus0 > ugen1.1: at usbus1 > uhub1: on usbus1 > ses0 at ahciem0 bus 0 scbus7 target 0 lun 0 > ses0: SEMB S-E-S 2.00 device > ses0: SEMB SES Device > ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 > ada0: ACS-2 ATA SATA 3.x device > ada0: Serial Number CVCV3102050X180EGN > ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 171705MB (351651888 512 byte sectors) > ada0: quirks=0x1<4K> > random: unblocking device. > Sending DHCP Discover packet from interface igb0 (00:25:90:9b:73:2e) > Sending DHCP Discover packet from interface igb1 (00:25:90:9b:73:2f) > Sending DHCP Discover packet from interface ix0 (90:e2:ba:77:d4:9c) > Sending DHCP Discover packet from interface ix1 (90:e2:ba:77:d4:9d) > ix0: link state changed to UP > ix1: link state changed to UP > uhub1: 2 ports with 2 removable, self powered > uhub0: 2 ports with 2 removable, self powered > ugen1.2: at usbus1 > uhub2: on usbus1 > ugen0.2: at usbus0 > uhub3: on usbus0 > uhub3: 6 ports with 6 removable, self powered > uhub2: 8 ports with 8 removable, self powered > ugen0.3: at usbus0 > ukbd0: on usbus0 > ums0: on usbus0 > ums0: 3 buttons and [Z] coordinates ID=0 > igb0: link state changed to UP > Received DHCP Offer packet on igb0 from 192.168.0.2 (accepted) (no root path) (boot_file) > Received DHCP Offer packet on igb0 from 192.168.0.2 (ignored) (no root path) (boot_file) > Received DHCP Offer packet on igb0 from 192.168.0.2 (ignored) (no root path) (boot_file) > Sending DHCP Request packet from interface igb0 (00:25:90:9b:73:2e) > Received DHCP Ack packet on igb0 from 192.168.0.2 (accepted) (got root path) > DHCP timeout for interface igb1 > DHCP timeout for interface ix0 > DHCP timeout for interface ix1 > Wired loader interface (IP 192.168.11.1) is igb0 > igb0 at 192.168.11.1 server 192.168.0.2 boot file /pxe/pxelinux.0 > subnet mask 255.255.0.0 router 192.168.0.2 rootfs 192.168.0.10:/muclab/image/machines/phobos2 hostname phobos2 > Adjusted interface igb0 > Shutdown interface igb1 > Shutdown interface ix0 > ix0: link state changed to DOWN > Shutdown interface ix1 > ix1: link state changed to DOWN > SMP: AP CPU #31 Launched! > SMP: AP CPU #10 Launched! > SMP: AP CPU #25 Launched! > SMP: AP CPU #14 Launched! > SMP: AP CPU #30 Launched! > SMP: AP CPU #12 Launched! > SMP: AP CPU #17 Launched! > SMP: AP CPU #7 Launched! > SMP: AP CPU #28 Launched! > SMP: AP CPU #13 Launched! > SMP: AP CPU #24 Launched! > SMP: AP CPU #6 Launched! > SMP: AP CPU #27 Launched! > SMP: AP CPU #8 Launched! > SMP: AP CPU #29 Launched! > SMP: AP CPU #11 Launched! > SMP: AP CPU #20 Launched! > SMP: AP CPU #15 Launched! > SMP: AP CPU #26 Launched! > SMP: AP CPU #9 Launched! > SMP: AP CPU #22 Launched! > SMP: AP CPU #5 Launched! > SMP: AP CPU #18 Launched! > SMP: AP CPU #4 Launched! > SMP: AP CPU #21 Launched! > SMP: AP CPU #16 Launched! > SMP: AP CPU #3 Launched! > SMP: AP CPU #1 Launched! > SMP: AP CPU #23 Launched! > SMP: AP CPU #2 Launched! > SMP: AP CPU #19 Launched! > Timecounter "TSC" frequency 2000045308 Hz quality 1000 > hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/4/48/0x3ff IAF/3/48/0x67 > Trying to mount root from nfs: []... > NFS ROOT: 192.168.0.10:/muclab/image/machines/phobos2 > ixl0: mem 0xdc800000-0xdcffffff,0xdd808000-0xdd80ffff irq 32 at device 0.0 on pci3 > ixl0: Using MSIX interrupts with 33 vectors > ixl0: f4.40 a1.4 n04.53 e80001dca > ixl0: Using defaults for TSO: 65518/35/2048 > ixl0: Ethernet address: 68:05:ca:32:15:d0 > ixl0: PCI Express Bus: Speed 8.0GT/s Width x8 > queues is 0xfffffe0008d03000 > ixl0: netmap queues/slots: TX 32/1024, RX 32/1024 > ixl1: mem 0xdc000000-0xdc7fffff,0xdd800000-0xdd807fff irq 32 at device 0.1 on pci3 > ixl1: Using MSIX interrupts with 33 vectors > ixl1: f4.40 a1.4 n04.53 e80001dca > ixl1: Using defaults for TSO: 65518/35/2048 > ixl1: Ethernet address: 68:05:ca:32:15:d1 > ixl1: PCI Express Bus: Speed 8.0GT/s Width x8 > queues is 0xfffffe0009227000 > ixl1: netmap queues/slots: TX 32/1024, RX 32/1024 > ix0: link state changed to UP > ix1: link state changed to UP > ixl0: link state changed to UP > ixl1: link state changed to UP > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------