Date: Sun, 18 Dec 2011 13:14:39 -0600 From: Alan Cox <alc@rice.edu> To: Sean Bruno <seanbru@yahoo-inc.com> Cc: Alan Cox <alc@cs.rice.edu>, "xen@freebsd.org" <xen@freebsd.org> Subject: Re: PV i386 patch Message-ID: <4EEE3B9F.2000104@rice.edu> In-Reply-To: <1324169806.3093.3.camel@hitfishpass-lx.corp.yahoo.com> References: <4EEB9CCE.6090701@rice.edu> <1324169806.3093.3.camel@hitfishpass-lx.corp.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/17/2011 18:56, Sean Bruno wrote:
> On Fri, 2011-12-16 at 11:32 -0800, Alan Cox wrote:
>> Is anyone here actively working on fixing problems with SMP support
>> under PV i386? While doing some other maintenance on the
>> vm_page_alloc() callers in the source tree, I happened to take a look at
>> cpu_initialize_context() in mp_machdep.c. This function is involved in
>> bringing up the 2nd, 3rd, etc. CPUs on an SMP system. I spotted a
>> couple obvious errors. First, the size parameter given to kmem_*()
>> functions is expected to be in terms of bytes and not pages. Second, I
>> believe that PV i386 requires PAE to be used. If so, there are out of
>> range accesses to the array m[].
>>
>> Index: i386/xen/mp_machdep.c
>> ===================================================================
>> --- i386/xen/mp_machdep.c (revision 228561)
>> +++ i386/xen/mp_machdep.c (working copy)
>> @@ -810,7 +810,7 @@ cpu_initialize_context(unsigned int cpu)
>> {
>> /* vcpu_guest_context_t is too large to allocate on the stack.
>> * Hence we allocate statically and protect it with a lock */
>> - vm_page_t m[4];
>> + vm_page_t m[NPGPTD + 2];
>> static vcpu_guest_context_t ctxt;
>> vm_offset_t boot_stack;
>> vm_offset_t newPTD;
>> @@ -831,8 +831,8 @@ cpu_initialize_context(unsigned int cpu)
>> pmap_zero_page(m[i]);
>>
>> }
>> - boot_stack = kmem_alloc_nofault(kernel_map, 1);
>> - newPTD = kmem_alloc_nofault(kernel_map, NPGPTD);
>> + boot_stack = kmem_alloc_nofault(kernel_map, PAGE_SIZE);
>> + newPTD = kmem_alloc_nofault(kernel_map, NPGPTD * PAGE_SIZE);
>> ma[0] = VM_PAGE_TO_MACH(m[0])|PG_V;
>>
>> #ifdef PAE
>> @@ -854,7 +854,7 @@ cpu_initialize_context(unsigned int cpu)
>> nkpt*sizeof(vm_paddr_t));
>>
>> pmap_qremove(newPTD, 4);
>> - kmem_free(kernel_map, newPTD, 4);
>> + kmem_free(kernel_map, newPTD, 4 * PAGE_SIZE);
>> /*
>> * map actual idle stack to boot_stack
>> */
>
> This seems happy on our ref9 VMs. I don't suppose this means I can go
> above 768M of Ram now?
It's not clear to me that this test actually exercised the function that
I changed, which is only executed when you spin up a 2nd, 3rd, etc.
processor, and it runs only on those processors. Am I missing something?
Alan
>
> [root@xen1 sbruno]# /usr/sbin/xm create -c ref9-xen32
> Using config file "/etc/xen/ref9-xen32".
> Started domain ref9-xen32 (id=106)
> WARNING: loader(8) metadata is missing!
> GDB: no debug ports present
> KDB: debugger backends: ddb
> KDB: current backend: ddb
> Copyright (c) 1992-2011 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights
> reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 9.0-PRERELEASE #0: Sat Dec 17 16:13:02 PST 2011
>
> sbruno@ref9-xen32.freebsd.org:/dumpster/scratch/sbruno-scratch/9/sys/i386/compile/XEN i386
> WARNING: WITNESS option enabled, expect reduced performance.
> Xen reported: 2493.756 MHz processor.
> Timecounter "ixen" frequency 1953125 Hz quality 0
> CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2493.76-MHz
> 686-class CPU)
> Origin = "GenuineIntel" Id = 0x10676 Family = 6 Model = 17
> Stepping = 6
>
> Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>
> Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
> AMD Features=0x20100000<NX,LM>
> AMD Features2=0x1<LAHF>
> real memory = 805306368 (768 MB)
> avail memory = 776830976 (740 MB)
> [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0)
> [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1)
> [XEN] xen_rtc_probe: probing Hypervisor RTC clock
> rtc0:<Xen Hypervisor Clock> on motherboard
> [XEN] xen_rtc_attach: attaching Hypervisor RTC clock
> xenstore0:<XenStore> on motherboard
> xc0:<Xen Console> on motherboard
> Event timer "ixen" quality 600
> Timecounters tick every 10.000 msec
> xenbusb_front0:<Xen Frontend Devices> on xenstore0
> [XEN] hypervisor wallclock nudged; nudging TOD.
> xn0:<Virtual Network Interface> at device/vif/0 on xenbusb_front0
> xn0: Ethernet address: 00:16:3e:00:00:03
> xenbusb_back0:<Xen Backend Devices> on xenstore0
> xctrl0:<Xen Control Device> on xenstore0
> xn0: backend features: feature-sg feature-gso-tcp4
> xbd0: 10240MB<Virtual Block Device> at device/vbd/768 on xenbusb_front0
> xbd0: attaching as ad0
> Timecounter "TSC" frequency 2493756000 Hz quality 800
> WARNING: WITNESS option enabled, expect reduced performance.
> Trying to mount root from ufs:/dev/ad0p2 []...
> rtc0: [XEN] xen_rtc_gettime
> rtc0: [XEN] xen_rtc_gettime: wallclock 1313550543 sec; 871707442 nsec
> rtc0: [XEN] xen_rtc_gettime: uptime 10619933 sec; 620343100 nsec
> rtc0: [XEN] xen_rtc_gettime: TOD 1324170477 sec; 492050542 nsec
> Setting hostuuid: 1c127834-ab5a-c2e4-7b24-5ea29d364d9d.
> Setting hostid: 0xdea9fbfd.
> Entropy harvesting: interrupts ethernet point_to_point kickstart.
> Starting file system checks:
> /dev/ad0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS
> /dev/ad0p2: clean, 1874771 free (883 frags, 234236 blocks, 0.0%
> fragmentation)
> Mounting local file systems:.
> Setting hostname: ref9-xen32.freebsd.org.
> xn0: link state changed to DOWN
> xn0: link state changed to UP
> Starting Network: lo0 xn0.
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> options=3<RXCSUM,TXCSUM>
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
> inet 127.0.0.1 netmask 0xff000000
> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=503<RXCSUM,TXCSUM,TSO4,LRO>
> ether 00:16:3e:00:00:03
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet manual
> status: active
> Starting devd.
> DHCPDISCOVER on xn0 to 255.255.255.255 port 67 interval 4
> DHCPOFFER from 69.147.83.62
> DHCPREQUEST on xn0 to 255.255.255.255 port 67
> DHCPACK from 69.147.83.62
> bound to 69.147.83.99 -- renewal in 900 seconds.
> add net ::ffff:0.0.0.0: gateway ::1
> add net ::0.0.0.0: gateway ::1
> add net fe80::: gateway ::1
> add net ff02::: gateway ::1
> Waiting 30s for the default route
> interface: .............................
> Mounting NFS file systems:.
> ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
> a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
> Creating and/or trimming log files.
> Starting syslogd.
> No core dumps found.
> Setting date via ntp.
> rtc0: [XEN] xen_rtc_settime
> 17 Dec 16:55:16 ntpdate[703]: step time server 69.147.83.54 offset
> -29601.428874 sec
> NFS access cache time=60
> Setting NIS domain: freebsd.
> Starting rpcbind.
> Starting ypbind.
> Clearing /tmp (X related).
> Updating motd:.
> Starting ntpd.
> Starting sshd.
> Starting cron.
> Starting background file system checks in 60 seconds.
>
> Sat Dec 17 16:55:17 PST 2011
>
> FreeBSD/i386 (ref9-xen32.freebsd.org) (xc0)
>
> login:
>
>
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4EEE3B9F.2000104>