Date: Sun, 29 Mar 2015 23:25:14 -0700 (PDT) From: Don Lewis <truckman@FreeBSD.org> To: alc@rice.edu Cc: src-committers@FreeBSD.org, alc@FreeBSD.org, svn-src-all@FreeBSD.org, bdrewery@FreeBSD.org, portmgr@FreeBSD.org, svn-src-head@FreeBSD.org, clusteradm@FreeBSD.org Subject: Re: svn commit: r280327 - in head/sys: kern vm Message-ID: <201503300625.t2U6PE3c093114@gw.catspoiler.org> In-Reply-To: <5516EB31.7000309@rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On 28 Mar, Alan Cox wrote: > On 03/28/2015 12:29, Bryan Drewery wrote: >> On 3/27/2015 9:41 PM, Don Lewis wrote: >>> On 21 Mar, Alan Cox wrote: >>>> Author: alc >>>> Date: Sat Mar 21 17:56:55 2015 >>>> New Revision: 280327 >>>> URL: https://svnweb.freebsd.org/changeset/base/280327 >>>> >>>> Log: >>>> Introduce vm_object_color() and use it in mmap(2) to set the color of >>>> named objects to zero before the virtual address is selected. Previously, >>>> the color setting was delayed until after the virtual address was >>>> selected. In rtld, this delay effectively prevented the mapping of a >>>> shared library's code section using superpages. Now, for example, we see >>>> the first 1 MB of libc's code on armv6 mapped by a superpage after we've >>>> gotten through the initial cold misses that bring the first 1 MB of code >>>> into memory. (With the page clustering that we perform on read faults, >>>> this happens quickly.) >>>> >>>> Differential Revision: https://reviews.freebsd.org/D2013 >>>> Reviewed by: jhb, kib >>>> Tested by: Svatopluk Kraus (armv6) >>>> MFC after: 6 weeks >>>> >>>> Modified: >>>> head/sys/kern/kern_exec.c >>>> head/sys/vm/vm_fault.c >>>> head/sys/vm/vm_mmap.c >>>> head/sys/vm/vm_object.h >>>> head/sys/vm/vnode_pager.c >>> This change appears to have partially broken package building. >>> >>> I recently set up a package building machine running 11.0-CURRENT >>> r280642 and found that it was unable to build openjdk7 packages inside >>> FreeBSD 8.4 and 9.3 poudriere jails, for both i386 and amd64. >>> >>> [snip] >>> Compiling ../generated/adfiles/ad_x86_64_gen.cpp >>> rm -f ad_x86_64_gen.o >>> c++ -D_ALLBSD_SOURCE -D_GNU_SOURCE -DAMD64 -DPRODUCT -I. -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/precompiled -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/cpu/x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os_cpu/bsd_x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/bsd/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_RELEASE_VERSION="\"24.76-b04\"" -DHOTSPOT_BUILD_TARGET="\"product\"" -DHOTSPOT_BUILD_USER="\"root\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"OpenJDK\"" -O2 -pipe -fstack-protector -fno-strict-aliasing -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 ! -fno! >>> -rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fPIC -fno-rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -O3 -fno-strict-aliasing -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-omit-frame-pointer -DINCLUDE_TRACE=1 -Wpointer-arith -Wconversion -Wsign-compare -c -fpch-deps -MMD -MP -MF ../generated/dependencies/ad_x86_64_format.o.d -o ad_x86_64_format.o ../generated/adfiles/ad_x86_64_format.cpp >>> c++ -D_ALLBSD_SOURCE -D_GNU_SOURCE -DAMD64 -DPRODUCT -I. -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/precompiled -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/cpu/x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os_cpu/bsd_x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/bsd/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_RELEASE_VERSION="\"24.76-b04\"" -DHOTSPOT_BUILD_TARGET="\"product\"" -DHOTSPOT_BUILD_USER="\"root\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"OpenJDK\"" -O2 -pipe -fstack-protector -fno-strict-aliasing -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 ! -fno! >>> -rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fPIC -fno-rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -O3 -fno-strict-aliasing -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-omit-frame-pointer -DINCLUDE_TRACE=1 -Wpointer-arith -Wconversion -Wsign-compare -c -fpch-deps -MMD -MP -MF ../generated/dependencies/ad_x86_64_gen.o.d -o ad_x86_64_gen.o ../generated/adfiles/ad_x86_64_gen.cpp >>> Compiling ../generated/adfiles/ad_x86_64_misc.cpp >>> rm -f ad_x86_64_misc.o >>> c++ -D_ALLBSD_SOURCE -D_GNU_SOURCE -DAMD64 -DPRODUCT -I. -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/precompiled -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/cpu/x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os_cpu/bsd_x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/bsd/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_RELEASE_VERSION="\"24.76-b04\"" -DHOTSPOT_BUILD_TARGET="\"product\"" -DHOTSPOT_BUILD_USER="\"root\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"OpenJDK\"" -O2 -pipe -fstack-protector -fno-strict-aliasing -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 ! -fno! >>> -rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fPIC -fno-rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -O3 -fno-strict-aliasing -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-omit-frame-pointer -DINCLUDE_TRACE=1 -Wpointer-arith -Wconversion -Wsign-compare -c -fpch-deps -MMD -MP -MF ../generated/dependencies/ad_x86_64_misc.o.d -o ad_x86_64_misc.o ../generated/adfiles/ad_x86_64_misc.cpp >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/compiler/abstractCompiler.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/utilities/accessFlags.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'abstractCompiler.o' failed >>> gmake[6]: *** [abstractCompiler.o] Error 1 >>> gmake[6]: *** Waiting for unfinished jobs.... >>> ../generated/adfiles/ad_x86_64.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >> Are you using ccache? >> >>> ../generated/adfiles/ad_x86_64_clone.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'accessFlags.o' failed >>> gmake[6]: *** [accessFlags.o] Error 1 >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_clone.o' failed >>> gmake[6]: *** [ad_x86_64_clone.o] Error 1 >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64.o' failed >>> gmake[6]: *** [ad_x86_64.o] Error 1 >>> ../generated/adfiles/ad_x86_64_expand.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_expand.o' failed >>> gmake[6]: *** [ad_x86_64_expand.o] Error 1 >>> ../generated/adfiles/ad_x86_64_gen.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> ../generated/adfiles/ad_x86_64_misc.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_gen.o' failed >>> gmake[6]: *** [ad_x86_64_gen.o] Error 1 >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_misc.o' failed >>> gmake[6]: *** [ad_x86_64_misc.o] Error 1 >>> ../generated/adfiles/ad_x86_64_format.cpp:1: fatal error: had to relocate PCH >>> compilation terminated. >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_format.o' failed >>> gmake[6]: *** [ad_x86_64_format.o] Error 1 >>> gmake[6]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/build/bsd-amd64/hotspot/outputdir/bsd_amd64_compiler2/product' >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/top.make:128: recipe for target 'the_vm' failed >>> gmake[5]: *** [the_vm] Error 2 >>> gmake[5]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/build/bsd-amd64/hotspot/outputdir/bsd_amd64_compiler2/product' >>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/Makefile:292: recipe for target 'product' failed >>> gmake[4]: *** [product] Error 2 >>> gmake[4]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/build/bsd-amd64/hotspot/outputdir' >>> Makefile:191: recipe for target 'generic_build2' failed >>> gmake[3]: *** [generic_build2] Error 2 >>> gmake[3]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make' >>> Makefile:151: recipe for target 'product' failed >>> gmake[2]: *** [product] Error 2 >>> gmake[2]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make' >>> make/hotspot-rules.gmk:111: recipe for target 'hotspot-build' failed >>> gmake[1]: *** [hotspot-build] Error 2 >>> gmake[1]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk' >>> Makefile:251: recipe for target 'build_product_image' failed >>> gmake: *** [build_product_image] Error 2 >>> ===> Compilation failed unexpectedly. >>> >>> I was not seeing this problem on my older package builder running >>> 10.1-STABLE. Since this problem has not shown up on the FreeBSD package >>> building cluster, I got suspicious that the change was quite recent. >>> >>> This old gcc bug report: >>> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=14940>; led me to suspect >>> mmap(). >>> >>> The old gcc source file /usr/src/contrib/gcc/ggc-common.c does a couple >>> of mmap() calls. Tne first is in mmap_gt_pch_get_address() where a >>> NULL first argument is used. The address that gets returned is stashed >>> away and the region is unmapped. Then a later call in >>> mmap_gt_pch_use_address() passes this saved address to mmap() as a >>> hint. It expects the mapped region to get mapped to the same base >>> address. If this does not happen, the above error is the result. >> I don't know what I'm talking about but that doesn't sound like a very >> safe assumption for the gcc code to make. > > > Your intuition is correct. It is not. And, in fact, under Solaris and > Linux, gcc does not make this assumption. > > I suspect that the solution used by gcc under Solaris would work for > us. However, that would entail modifying gcc in older branches. After digging in to this today, I'm not sure that the Solaris solution would work. On amd64, with an amd64 jail, and kernel rev r280326, I observe the following: One gcc process calls mmap() with addr=0 and len=0x657a000, and the value 0x811400000 is returned. Subsequent gcc processes call mmap() with addr=0x811400000 and len=0x657a000, getting 0x811400000 in return. With kernel rev r280327 I see: One gcc process calls mmap() with addr=0 and len=0x657a000, and the value 0x811400000 is returned. Subsequent gcc processes call mmap() with addr=0x811400000 and len=0x657a000, getting 0x8115f4000 in return. What I later noticed is that the subsequent calls are passing offset=0x1f4000. Not so coincidentally, 0x811400000 + 0x1f4000 = 0x8115F4000. My first attempt at a fix subtracted offset from address, but the mmap() return value changed to 0x8113f4000 instead of the 0x811400000 I was expecting. It looked to me like the code must be doing superpage alignment on the start of the file and then adding the offset to get the start of the mapped region. This somewhat hacking patch disables this alignment if a non-zero address is passed as a hint, and allows the code to make the start of the mapped region match the hint. With this patch, I've been able to build openjdk7 in a FreeBSD 9.3 amd64 jail. Index: vm_mmap.c =================================================================== --- vm_mmap.c (revision 280327) +++ vm_mmap.c (working copy) @@ -325,6 +325,8 @@ * There should really be a pmap call to determine a reasonable * location. */ + if (addr != 0 && (flags & MAP_ANON) == 0) + flags |= MAP_RESERVED0100; PROC_LOCK(td->td_proc); if (addr == 0 || (addr >= round_page((vm_offset_t)vms->vm_taddr) && @@ -1665,6 +1667,8 @@ else if ((flags & MAP_ALIGNMENT_MASK) != 0) findspace = VMFS_ALIGNED_SPACE(flags >> MAP_ALIGNMENT_SHIFT); + else if ((flags & MAP_RESERVED0100) != 0) + findspace = VMFS_ANY_SPACE; else findspace = VMFS_OPTIMAL_SPACE; rv = vm_map_find(map, object, foff, addr, size,
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201503300625.t2U6PE3c093114>