Date: Mon, 21 Nov 2011 23:19:08 +0100 From: Olivier Smedts <olivier@gid0.org> To: "Conrad J. Sabatier" <conrads@cox.net> Cc: freebsd-ports@freebsd.org Subject: Re: Can't compile kde4 and kdelibs4 with an uptodate amd64 Releng machine. ( Message-ID: <CABzXLYOR_LD4eZXrOye8_OFtMBR=n9htjdDM6iLbBgWgrwg3Vw@mail.gmail.com> In-Reply-To: <20111121153828.4e04a92a@cox.net> References: <20111121074243.18902mpt8znqto40@econet.encontacto.net> <20111121100945.2c888eaf@cox.net> <CABzXLYNF6T-GzOVL9OBjPKfOfb9xjkN4_ffa=V=rZma4OdBVHQ@mail.gmail.com> <20111121153828.4e04a92a@cox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
2011/11/21 Conrad J. Sabatier <conrads@cox.net>:
> On Mon, 21 Nov 2011 17:27:42 +0100
> Olivier Smedts <olivier@gid0.org> wrote:
>
>> 2011/11/21 Conrad J. Sabatier <conrads@cox.net>:
>> > On Mon, 21 Nov 2011 07:42:43 -0600
>> > eculp <eculp@encontacto.net> wrote:
>> >>
>> >> I have tried building both from the different ports and even more
>> >> using portmaster and all stop ate similar locations in kdelabs4.
>> >> Maybe there is something that I should or could build first.
>> >>
>> >> errors follow:
>> >>
>> >> kdelibs stops here:
>> >>
>> >> Generating kurlnavigator.moc
>> >> Generating moc_kdiroperatordetailview_p.cpp
>> >> [ =A01%] Built target kfile_automoc
>> >> Scanning dependencies of target kdeinit_kconf_update_automoc
>> >> Scanning dependencies of target docbookl10nhelper_automoc
>> >> [ =A01%] Built target kdeinit_kconf_update_automoc
>> >> [ =A01%] Built target docbookl10nhelper_automoc
>> >> Scanning dependencies of target genshortcutents_automoc
>> >> Scanning dependencies of target kio_ghelp_automoc
>> >> [ =A01%] Built target genshortcutents_automoc
>> >> [ =A01%] Built target kio_ghelp_automoc
>> >> Scanning dependencies of target kio_help_automoc
>> >> Scanning dependencies of target meinproc4_automoc
>> >> [ =A01%] Built target kio_help_automoc
>> >> [ =A01%] Built target meinproc4_automoc
>> >> Scanning dependencies of target meinproc4_simple_automoc
>> >> Scanning dependencies of target kio_file_automoc
>> >> [ =A01%] Built target meinproc4_simple_automoc
>>
>> I've got this behavior on the 2 desktop machines I use.
>>
>> >> kdepimlibs4 stops here:
>> >>
>> >> Scanning dependencies of target kimg_pcx_automoc
>> >> Scanning dependencies of target kimg_pic_automoc
>> >> [ =A01%] Built target kimg_pcx_automoc
>> >> [ =A01%] Built target kimg_pic_automoc
>> >> Scanning dependencies of target kimg_psd_automoc
>> >> Scanning dependencies of target kimg_ras_automoc
>> >> [ =A01%] Built target kimg_psd_automoc
>> >> [ =A01%] Built target kimg_ras_automoc
>> >> Scanning dependencies of target kimg_rgb_automoc
>> >> Scanning dependencies of target kimg_tga_automoc
>> >> [ =A01%] Built target kimg_rgb_automoc
>> >> [ =A01%] Built target kimg_tga_automoc
>> >> Scanning dependencies of target kimg_xcf_automoc
>> >> Scanning dependencies of target kimg_xview_automoc
>> >> [ =A01%] Built target kimg_xview_automoc
>> >> [ =A01%] Built target kimg_xcf_automoc
>> >> Scanning dependencies of target kdnssd_automoc
>> >> Scanning dependencies of target krosscore_automoc
>> >> Generating interpreter.moc
>> >> Generating script.moc
>> >> Generating action.moc
>> >> Generating actioncollection.moc
>> >> Generating manager.moc
>> >> [ =A01%] Built target krosscore_automoc
>> >
>> > So where are the errors? =A0There are none in the output you posted.
>>
>> I think there's no error (if it's the same problem as mine).
>>
>> For me, the build process seems to stop/freeze randomly, most often
>> after "Built target XXX". It affects only KDE ports, no other
>> qt4-qmake or cmake consumer. No CPU usage. No disk usage. No excessive
>> or changing memory usage... I didn't report it earlier because I don't
>> know how to debug this, and it did not seem to affect other users
>> (until now).
>
> OK, I didn't get that point from the original poster. =A0I was looking to
> see some actual error output.
>
>> Here is the "workaround" I painfully used on my 2 desktop machines :
>>
>> # cd /usr/ports/x11/kde4
>> # make
>> wait for a freeze...
>> ^C
>> # make
>> wait for a freeze...
>> ^C
>> # make
>> ...
>> I maybe had to restart the build one or two hundred times to have a
>> fully installed KDE4.
>
> Wow, "painful" is an understatement here, to say the least. =A0:-)
>
> Have you tried using truss or ktrace to see what's going on when these
> "freezes" occur?
>
> You'll want to be sure to enable tracing descendents of the original
> make process as well. =A0Ports makes, as you no doubt are aware, spawn
> numerous processes along the way.
>
> truss -f make
> (or)
> ktrace -i make
>
> See the man pages for other options you may want to use as well.
>
> ktrace, in particular, will produce *copious* output. =A0You'll probably
> want to just do a "tail" on the generated ktrace.out file:
>
> kdump | tail -<some number of lines> | more
>
>> I've got this behavior since KDE 4.7.X (4.7.2 and 4.7.3), I had no
>> problems building KDE 4.6.X.
>>
>> I even tried deleting all ports, cleaning /usr/local, tried again. No
>> change. Tried compiling all ports with gcc instead of clang, no
>> change. Forced make jobs UNSAFE, no change.
>>
>> I use FreeBSD 9.0 amd64, system built with clang (are you ?).
>
> No, I only use the default system gcc:
>
> # gcc -v
> Using built-in specs.
> Target: amd64-undermydesk-freebsd
> Configured with: FreeBSD/amd64 system compiler
> Thread model: posix
> gcc version 4.2.1 20070831 patched [FreeBSD]
>
>> %cat /etc/make.conf
>> SVN_UPDATE=3Dyes
>> SVN=3D/usr/local/bin/svn
>> CPUTYPE?=3Dcore2
>
> I've been using the (undocumented, at least in /etc/make.conf)
> CPUTYPE?=3Dnative with no problems for quite some time now. =A0Let gcc
> detect the processor type and generate the appropriate code.
> Eliminates any guesswork in trying to select the correct setting for
> CPUTYPE.
CPUTYPE=3Dnative is not recognized by /usr/share/mk/bsd.cpu.mk (that's
the real purpose of CPUTYPE, it does not only change the -march
compiler setting).
The proper way of doing what you're doing, after numerous tests and
researchs, seems to be :
CPUTYPE?=3Dcore2 (for example, to let /usr/share/mk/bsd.cpu.mk do its job)
CFLAGS=3D-O2 -pipe -march=3Dnative (because you want the compiler to
detect the cpu it's running on and optimize the code for it)
NO_CPU_CFLAGS=3Dyes (because you wanted to force the -march, you don't
want another one to be added on the command line)
COPTFLAGS=3D-O2 -pipe -march=3Dnative (same thing for kernel CFLAGS)
NO_CPU_COPTFLAGS=3Dyes
This way, bsd.cpu.mk can set useful MACHINE_CPU for your CPUTYPE, but
you let the compiler determine which processor to optimize the code
for with the -march. I add NO_CPU_CFLAGS and NO_CPU_COPTFLAGS to be
able to specify -march=3Dnative in the CFLAGS, cause it's different from
CPUTYPE.
Now why do I force -march=3Dcore2 and don't use -march=3Dnative ? Because
our base gcc does not use the correct flags on my Core2 CPU if using
-march=3Dnative :
% /usr/bin/gcc -### -march=3Dnative md5.c
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
"/usr/libexec/cc1" "-quiet" "-D_LONGLONG" "md5.c" "-march=3Dcore2"
"-mtune=3Dgeneric" "-quiet" "-dumpbase" "md5.c" "-auxbase" "md5" "-o"
"/var/tmp//ccYJKvGN.s"
"/usr/bin/as" "-Qy" "-o" "/var/tmp//ccR6Lu5X.o" "/var/tmp//ccYJKvGN.s"
"/usr/bin/ld" "--eh-frame-hdr" "-dynamic-linker"
"/libexec/ld-elf.so.1" "/usr/lib/crt1.o" "/usr/lib/crti.o"
"/usr/lib/crtbegin.o" "-L/usr/lib" "-L/usr/lib" "/var/tmp//ccR6Lu5X.o"
"-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc"
"--as-needed" "-lgcc_s" "--no-as-needed" "/usr/lib/crtend.o"
"/usr/lib/crtn.o"
See the "-mtune=3Dgeneric" ? Crap ! You don't want that (manpage :
Produce code optimized for the most common IA32/AMD64/EM64T
processors. If you know the CPU on which your code will run, then you
should use the corresponding -mtune option instead of -mtune=3Dgeneric.
But, if you do not know exactly what CPU users of your application
will have, then you should use this option.)
% /usr/bin/gcc -### -march=3Dcore2 md5.c
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
"/usr/libexec/cc1" "-quiet" "-D_LONGLONG" "md5.c" "-quiet"
"-dumpbase" "md5.c" "-march=3Dcore2" "-auxbase" "md5" "-o"
"/var/tmp//ccL8Bvk4.s"
"/usr/bin/as" "-Qy" "-o" "/var/tmp//ccLrppPo.o" "/var/tmp//ccL8Bvk4.s"
"/usr/bin/ld" "--eh-frame-hdr" "-dynamic-linker"
"/libexec/ld-elf.so.1" "/usr/lib/crt1.o" "/usr/lib/crti.o"
"/usr/lib/crtbegin.o" "-L/usr/lib" "-L/usr/lib" "/var/tmp//ccLrppPo.o"
"-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc"
"--as-needed" "-lgcc_s" "--no-as-needed" "/usr/lib/crtend.o"
"/usr/lib/crtn.o"
No -mtune=3Dgeneric. According to the gcc manpage for the x86 arch,
-march=3Dcore2 is sufficient to have proper values for -mtune, -mcpu...
(Generate instructions for the machine type cpu-type. The choices for
cpu-type are the same as for -mtune. Moreover, specifying
-march=3Dcpu-type implies -mtune=3Dcpu-type.)
>> KERNCONF=3DCORE
>> CFLAGS=3D-O2 -pipe -march=3Dcore2 -fomit-frame-pointer
>
> There's no need to add -march=3D to CFLAGS, if you're setting CPUTYPE
> (that's what CPUTYPE is for).
Not really...
>> NO_CPU_CFLAGS=3Dyes
>
> Why are you setting CPUTYPE, and then telling make not to use it? =A0And
> then, setting the CPU type anyway in your CFLAGS? =A0:-)
>
>> COPTFLAGS=3D-O2 -pipe -march=3Dcorei7 -fomit-frame-pointer
>> NO_CPU_COPTFLAGS=3Dyes
>
> Again, same question as above.
>
> Even more pointedly, why core2 in CFLAGS and corei7 (what is that?) in
> COPTFLAGS?
Because clang is somewhat broken with -march=3Dcorei7, but the kernel
compiles fine with it. -march=3Dcore2 is the nearest arch working
correctly. On another computer (with a Core2 CPU this time, not a
Corei7) I use -march=3Dcore2 in both CFLAGS and COPTFLAGS.
>> BOOTWAIT=3D0
>> WITHOUT_PROFILE=3Dyes
>
> Yes, WITHOUT_PROFILE=3Dyes is the most sensible choice for most users.
> Should be enabled by default, IMHO.
>
>> .if !${.CURDIR:M/usr/ports/deskutils/kdepimlibs4*} &&
>> !${.CURDIR:M/usr/ports/devel/icu*} &&
>> !${.CURDIR:M/usr/ports/editors/kate*} &&
>> !${.CURDIR:M/usr/ports/games/kdegames4*} &&
>> !${.CURDIR:M/usr/ports/graphics/libwpg*} &&
>> !${.CURDIR:M/usr/ports/graphics/netpbm*} &&
>> !${.CURDIR:M/usr/ports/graphics/vigra*} &&
>> !${.CURDIR:M/usr/ports/multimedia/kdemultimedia4*} &&
>> !${.CURDIR:M/usr/ports/net/hupnp*} &&
>> !${.CURDIR:M/usr/ports/net/kdenetwork4*} &&
>> !${.CURDIR:M/usr/ports/textproc/libwpd*} &&
>> !${.CURDIR:M/usr/ports/textproc/libwps*} &&
>> !${.CURDIR:M/usr/ports/www/firefox*} &&
>> !${.CURDIR:M/usr/ports/www/libxul*} &&
>> !${.CURDIR:M/usr/ports/www/qt4-webkit*} &&
>> !${.CURDIR:M/usr/ports/x11/kde4-baseapps*} &&
>> !${.CURDIR:M/usr/ports/x11/kde4-runtime*} &&
>> !${.CURDIR:M/usr/ports/x11/kde4-workspace*} &&
>> !${.CURDIR:M/usr/ports/x11/kdelibs4*}
>> .if !defined(CC) || ${CC} =3D=3D "cc"
>> CC=3Dclang
>> .endif
>> .if !defined(CXX) || ${CXX} =3D=3D "c++"
>> CXX=3Dclang++
>> .endif
>> .if !defined(CPP) || ${CPP} =3D=3D "cpp"
>> CPP=3Dclang -E
>> .endif
>
> Hmmm. =A0Could it be that your problem is related to your selective use
> of clang instead of gcc for certain ports (combined with using clang
> for the base system/kernel)?
I don't think, because I previously tried with gcc for all ports.
>> NO_WERROR=3D
>> WERROR=3D
>
> What is your intention in unsetting these last two variables? =A0If the
> idea is to ensure that warnings will never cause an error to be
> generated, you probably want to set both instead to:
>
> NO_WERROR=3D =A0 =A0 =A0-Wno-error
> WERROR=3D =A0 =A0 =A0 =A0 -Wno-error
>
>> .endif
See http://wiki.freebsd.org/BuildingFreeBSDWithClang
>> EXPLICIT_PACKAGE_DEPENDS=3Dyes
>> FORCE_MAKE_JOBS=3Dyes
>
> Have you tried unsetting this last variable to see if it stops these
> "freezes" from occurring?
Yes, I also tried forcing UNSAFE make jobs. Maybe less problems, but
still, numerous freezes when compiling kde ports.
>> WRKDIRPREFIX=3D/tmp
>
> Do you have any particular reason for not using the default for this
> variable?
Yes, my /tmp is a tmpfs. I also tried with the default WRKDIRPREFIX.
>> WITHOUT_X11R6_SYMLINK=3Dyes
>> NOPORTDOCS=3Dyes
>> NOPORTEXAMPLES=3Dyes
>> WITH_OPTIMIZED_CFLAGS=3Dyes
>> VIDEO_DRIVER=3Dati
>> WITHOUT_NOUVEAU=3Dyes
>> WITHOUT_HAL=3Dyes
>> WITHOUT_DBUS=3Dyes
>> WITHOUT_GCONF=3Dyes
>> WITHOUT_LIBNOTIFY=3Dyes
>> WITHOUT_AVAHI=3Dyes
>> WITH_CDROM=3D/dev/cd0
>> WITHOUT_SWITCHER=3Dyes
>> THUNDERBIRD_I18N=3Dfr
>> LOCALIZED_LANG=3Dfr
>> PERL_VERSION=3D5.10.1
>
> While setting all of these variables globally like this is most likely
> perfectly harmless, there *is* nonetheless a possibility of some
> unexpected side-effect that's going unnoticed in some of your ports
> builds.
Using that since ages. I only dumped the full contents of my make.conf
to see if eculp has something similar, for examble a clang-compiled
world.
But, to make it clearer :
# mv /etc/make.conf /etc/make.conf.old
# cd /usr/ports/x11/kdelibs4
=3D=3D=3D> Building for kdelibs-4.7.3
Scanning dependencies of target KDECMakeModulesManPage
[ 0%] Built target KDECMakeModulesManPage
Scanning dependencies of target kdeinit4_shutdown
Scanning dependencies of target kdesu_stub
Scanning dependencies of target kpac_dhcp_helper
Scanning dependencies of target kdefakes
[ 0%] Building C object kdesu/CMakeFiles/kdesu_stub.dir/kdesu_stub.o
[ 0%] Building C object kinit/CMakeFiles/kdeinit4_shutdown.dir/wrapper.o
[ 0%] Building C object
kio/misc/kpac/CMakeFiles/kpac_dhcp_helper.dir/kpac_dhcp_helper.o
[ 0%] Building C object kdecore/CMakeFiles/kdefakes.dir/fakes.o
Linking C shared library ../lib/libkdefakes.so
[ 0%] Built target kdefakes
Scanning dependencies of target kdeinit4_wrapper
[ 1%] Building C object kinit/CMakeFiles/kdeinit4_wrapper.dir/wrapper.o
Linking C executable ../bin/kdesu_stub
Linking C executable ../../../bin/kpac_dhcp_helper
[ 1%] Built target kdesu_stub
[ 1%] Built target kpac_dhcp_helper
Scanning dependencies of target kshell4
Scanning dependencies of target kwrapper4
Linking C executable ../bin/kdeinit4_shutdown
[ 1%] Built target kdeinit4_shutdown
[ 1%] Building C object kinit/CMakeFiles/kshell4.dir/shell.o
[ 1%] Building C object kinit/CMakeFiles/kwrapper4.dir/kwrapper.o
Linking C executable ../bin/kdeinit4_wrapper
Scanning dependencies of target start_kdeinit
[ 1%] Building C object kinit/CMakeFiles/start_kdeinit.dir/start_kdeinit.o
[ 1%] Built target kdeinit4_wrapper
Scanning dependencies of target start_kdeinit_wrapper
[ 1%] Building C object
kinit/CMakeFiles/start_kdeinit_wrapper.dir/start_kdeinit_wrapper.o
Linking C executable ../bin/start_kdeinit
Linking C executable ../bin/kshell4
Linking C executable ../bin/start_kdeinit_wrapper
Linking C executable ../bin/kwrapper4
[ 1%] Built target start_kdeinit
[ 1%] Built target start_kdeinit_wrapper
[ 1%] Built target kshell4
Scanning dependencies of target nepomuk_automoc
Scanning dependencies of target nepomuk-rcgen_automoc
[ 1%] Built target kwrapper4
Scanning dependencies of target nepomukquery_automoc
Scanning dependencies of target nepomukutils_automoc
Generating kmetadatatagcloud.moc
Generating queryserviceclient_p.moc
[ 1%] Built target nepomuk-rcgen_automoc
Generating queryserviceclient.moc
Generating ktagcloudwidget.moc
[ 1%] Built target nepomukquery_automoc
Generating ktagdisplaywidget.moc
Generating resourcefiltermodel.moc
Generating tagwidget.moc
Generating tagcheckbox.moc
Generating resourcemanager.moc
Generating nepomukmainmodel.moc
Generating nepomukmassupdatejob.moc
Generating nepomukservice.moc
Generating kedittagsdialog_p.moc
Generating graphwrapper_p.moc
[ 1%] Built target nepomuk_automoc
Scanning dependencies of target nepomuk_testbase_automoc
Scanning dependencies of target kauth-policy-gen_automoc
Generating testbase.moc
[ 1%] Built target kauth-policy-gen_automoc
[ 1%] Built target nepomuk_testbase_automoc
Scanning dependencies of target kauth_backend_plugin_automoc
Scanning dependencies of target kauth_helper_plugin_automoc
Generating AuthBackend.moc
Generating moc_Polkit1Backend.cpp
[ 1%] Built target kauth_backend_plugin_automoc
freeze...
"make" is PID 15300
# truss -f -p 15300
nothing happens
> (locale stuff snipped)
>
> Sorry I don't have any more useful help to provide. =A0Try tracing these
> "freezing" builds and see if anything "interesting" turns up.
Thanks for the suggestions. I'll try to have something useful with
truss and/or ktrace.
--=20
Olivier Smedts=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0 _
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 ASCII ribbon campaign ( )
e-mail: olivier@gid0.org=A0 =A0 =A0 =A0 - against HTML email & vCards=A0 X
www: http://www.gid0.org=A0 =A0 - against proprietary attachments / \
=A0 "Il y a seulement 10 sortes de gens dans le monde :
=A0 ceux qui comprennent le binaire,
=A0 et ceux qui ne le comprennent pas."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABzXLYOR_LD4eZXrOye8_OFtMBR=n9htjdDM6iLbBgWgrwg3Vw>