From owner-freebsd-java@freebsd.org Sat Feb 13 22:56:39 2016 Return-Path: Delivered-To: freebsd-java@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02A48AA80DC for ; Sat, 13 Feb 2016 22:56:39 +0000 (UTC) (envelope-from glewis@eyesbeyond.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E17651ED3 for ; Sat, 13 Feb 2016 22:56:38 +0000 (UTC) (envelope-from glewis@eyesbeyond.com) Received: by mailman.ysv.freebsd.org (Postfix) id DCCB0AA80D9; Sat, 13 Feb 2016 22:56:38 +0000 (UTC) Delivered-To: java@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC438AA80D8 for ; Sat, 13 Feb 2016 22:56:38 +0000 (UTC) (envelope-from glewis@eyesbeyond.com) Received: from misty.eyesbeyond.com (gerbercreations.com [71.39.140.16]) by mx1.freebsd.org (Postfix) with ESMTP id 8A5191ED2 for ; Sat, 13 Feb 2016 22:56:37 +0000 (UTC) (envelope-from glewis@eyesbeyond.com) Received: from misty.eyesbeyond.com (localhost.eyesbeyond.com [127.0.0.1]) by misty.eyesbeyond.com (8.15.2/8.15.2) with ESMTP id u1DMuaqB004425; Sat, 13 Feb 2016 14:56:36 -0800 (PST) (envelope-from glewis@eyesbeyond.com) Received: (from glewis@localhost) by misty.eyesbeyond.com (8.15.2/8.15.2/Submit) id u1DMuZJ6004424; Sat, 13 Feb 2016 14:56:35 -0800 (PST) (envelope-from glewis@eyesbeyond.com) X-Authentication-Warning: misty.eyesbeyond.com: glewis set sender to glewis@eyesbeyond.com using -f Date: Sat, 13 Feb 2016 14:56:35 -0800 From: Greg Lewis To: Brian Gardner Cc: Magnus Ihse Bursie , java@freebsd.org Subject: Re: Helping out with JDK 9 on BSD Message-ID: <20160213225635.GB64267@misty.eyesbeyond.com> References: <56ADE943.6020103@oracle.com> <8B9ACABA-F9B2-4288-9DB5-CA843DE48C34@getsnappy.com> <20160203034053.GA36170@misty.eyesbeyond.com> <56B45453.9020006@oracle.com> <1C39812E-E596-47E6-AB45-0A591A017FFB@getsnappy.com> <20160213080945.GC56357@misty.eyesbeyond.com> <99C45649-B498-4BE1-97D4-0CEE0BC8A77D@getsnappy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <99C45649-B498-4BE1-97D4-0CEE0BC8A77D@getsnappy.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-java@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting Java to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Feb 2016 22:56:39 -0000 I'm probably going to look at these piecemeal... On Sat, Feb 13, 2016 at 11:08:36AM -0800, Brian Gardner wrote: > I???m sending those links again, it came through weird on the mailing list > porting build_vm_def.sh from bsd-port/jdk8 repo, to fix NM errors during = build > http://brian.timestudybuddy.com/webrev/hotspot__NM/webrev/ This one looks straight forward. +1 > Add SUPPORT_RESERVED_STACK_AREA flag for all BSD's > http://brian.timestudybuddy.com/webrev/hotspot__SUPPORT_RESERVED_STACK_AR= EA/webrev/ > porting getthreadid logic from bsd-port/jdk8. calling syscall(SYS_thr_sel= f) caused pthread_setspecific to be cleared. > http://brian.timestudybuddy.com/webrev/hotspot__os_bsd_cpp__getthreadid/w= ebrev/ > adding in servicability agent ported from bsd-port/jdk8 > http://brian.timestudybuddy.com/webrev/hotspot__sa/webrev/ > adding classlist.bsd that is identical to classlist.linux, in order to co= mpile > http://brian.timestudybuddy.com/webrev/jdk__classlist-bsd/webrev/ This also looks straight forward. There are differences in openjdk8, but frankly the differences just make it look like they are out of sync, not that they meaningfully differ. +1 > clean up TLS current thread at end of ::run functions similar to how it's= done in openjdk8. > http://brian.timestudybuddy.com/webrev/hotspot__clear_thread_current/webr= ev/ > clear current thread before exiting java_start to avoid warnings from lef= tover pthread_setspecific data > http://brian.timestudybuddy.com/webrev/hotspot__clear_thread_current_alt/= webrev/ Will look at the others as I have time. >=20 > > On Feb 13, 2016, at 12:09 AM, Greg Lewis wrote: > >=20 > > On Fri, Feb 12, 2016 at 10:54:29AM -0800, Brian Gardner wrote: > >> After giving it some thought, I???ve created an alternate patch for th= e clear_current_thread issue. I like this solution better because it is is= olated to java_start function in os_bsd.cpp, and it makes java_start respon= sible for cleaning up the current_thread since it also initializes it. > >=20 > > Sounds good! What can I do to help get all the changes up for review? > >=20 > > -- Greg > >=20 > >>> On Feb 10, 2016, at 10:45 PM, Brian Gardner w= rote: > >>>=20 > >>> Hello again Greg and Magnus, > >>> I finished up researching the leftover thread-specific data after 4 d= estructor iterations issue. It looks like it???s currently functioning on = Freebsd the same way it does on Linux, the only difference is that Freebsd = output that warning were Linux silently stops after it???s forth try. I lo= oked into how openjdk8 was working and it looks like there were calls to TL= S::set_thread(NULL) scattered throughout the code base at the end of some := :run methods. I???ve mimicked that behavior in the necessary spots with th= e following patch. I can???t help but think there must be a better way to = get these Thread???s destructors called, and I tried that in place of makin= g clear_thread_current public and calling it directly, but it resulted in d= eadlock. > >>>=20 > >>> > >>>=20 > >>>> On Feb 9, 2016, at 2:36 PM, Brian Gardner > wrote: > >>>>=20 > >>>> Hi Greg and Magnus, > >>>> I???ve attached the patches needed to build and compile on Freebsd. = I???ve reverted my changes to TLS, and done more debugging there. There a= ppears to be two separate issues. One issue was that the implementation of= OS::bad::gettid() was somehow clearing the current thread previously set w= ith pthread_setspecific. I was able to fix this by porting logic from bad-= port/jdk8, see hotspot__os_bd_cpp__getthreadid.patch. The second issue see= ms to be related to certain types of threads never getting their destructor= called and therefor not clearing pthread_setspecific and resulting in Thre= ad 803c59000 has exited with leftover thread-specific data after 4 destruct= or iterations, errors. This one I haven???t tracked down yet and will cont= inue looking into. While I can compile and execute HelloWorld, I???m getti= ng a ton of these benign yet annoying messages from ConcurrentGC and Worker= threads. I???ll continue looking into that next week. > >>>>=20 > >>>>=20 > >>>>> On Feb 4, 2016, at 11:50 PM, Magnus Ihse Bursie > wrote: > >>>>>=20 > >>>>> On 2016-02-04 01:29, Brian Gardner wrote: > >>>>>> Hi Greg, > >>>>>> Great to hear from you, it???s been a while since I took on one of= these projects, but I do follow the java@freebsd mailing list and see all = the work you put in. I must tell you how greatly your work is appreciated b= y me and I???m sure the rest of the community. I have good news, with fair= ly minimal changes I have it compiling, building HelloWorld, and running He= lloWorld all without errors. > >>>>>>=20 > >>>>>> I think it???ll break down into 4 change sets, > >>>>>> NM - I ported some code from bsd-port/jdk8/hotspot/make/bsd/makefi= les/build_vm_def.sh that fixes a compile error=20 > >>>>>> SUPPORT_RESERVED_STACK_AREA - define SUPPORT_RESERVED_STACK_AREA f= or _ALLBSD_SOURCE > >>>>>> Serviceability Agent - ported from bad-port/jdk8, currently compil= es but hasn???t been tested > >>>>>> TLS - there is problems with the TLS in jdk9 used for Thread::curr= ent(). This caused several issues for me and while the fix was relatively = straight forward in using THREAD_LOCAL_DECL completely in place of TLS comp= letely, I???d imagine there was a reason it was implemented in sort of a Hy= brid fashion alongside TLS. > >>>>>=20 > >>>>> There is a reason. It is very well described in this comment: > >>>>> https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommentId= =3D13866505 > > >>>>>=20 > >>>>> The TL;DR: in some circumstances the JVM can crash if we do not set= up an initial TLS. On the other hand, this was for glibc, so maybe this si= tuation does not arise for FreeBSD. > >>>>>=20 > >>>>>>=20 > >>>>>> It would be great if I could post these as web reviews for others = to review. Greg, would getting access to bsd-port/jdk9 allow us to collabo= rate through web reviews? =20 > >>>>>=20 > >>>>>=20 > >>>>>> Magnus, do you think using bsd-port/jdk9 for this purpose be a ste= p towards getting these basic changes into the jdk9 mainline, or would your= colleagues think "if they have bsd-port/jdk9, why bother with integrating = them into the jdk9 mainline???? > >>>>>=20 > >>>>> I think putting these changes in bsd-port/jdk9 is very reasonable, = and I do not believe it will in any way hinder their acceptance into jdk9 m= ainline. The only thing to be careful about there, though, is that all cont= ributors have signed the OCA (Oracle contributor agreement). Mixing in "leg= ally bad" code can present a real problem for adoption into mainline. > >>>>>=20 > >>>>> /Magnus > >>>>>=20 > >>>>>>=20 > >>>>>> Brian Gardner > >>>>>>=20 > >>>>>>> On Feb 2, 2016, at 7:40 PM, Greg Lewis < >glewis@eyesbeyond.com = >> wrote: > >>>>>>>=20 > >>>>>>> On Sun, Jan 31, 2016 at 06:57:26AM -0800, Brian Gardner wrote: > >>>>>>>> I???m interested in helping. I ported openjdk6 to freebsd and a= lso helped out with openjdk8. > >>>>>>>>=20 > >>>>>>>> Brian Gardner > >>>>>>>=20 > >>>>>>> I'd love to see as much of these changes get into the jdk9 mainli= ne. My time > >>>>>>> is really limited for the next couple of weeks though. If it wou= ld help you > >>>>>>> or Magnus to use the bsd-port repo to stage changes then that wou= ld be > >>>>>>> great too. Let me do a merge to make sure it is up to date with = mainline. > >>>>>>> Notionally we'd have to do a vote to get you both access I think?= But I > >>>>>>> can shepherd changes in if that will help with sharing them. > >>>>>>>=20 > >>>>>>> FWIW, in terms of taking responsibility, I've been regularly upda= ting the > >>>>>>> repos for jdk8 and jdk7 for a couple of years. I'm happy to keep= jdk9 > >>>>>>> building if that is what is being looked for there and also port = more of > >>>>>>> the BSD changes for jdk8 as appropriate. > >>>>>>>=20 > >>>>>>> - Greg > >>>>>>>=20 > >>>>>>>>> On Jan 31, 2016, at 3:00 AM, Magnus Ihse Bursie >> wrote: > >>>>>>>>>=20 > >>>>>>>>> On 2016-01-31 11:54, Magnus Ihse Bursie wrote: > >>>>>>>>>> Hi, > >>>>>>>>>>=20 > >>>>>>>>>> [TL;DR: I'm offering patches to compile JDK 9 on FreeBSD, but = it's not clear where to put them, or how.] > >>>>>>>>>>=20 > >>>>>>>>>> I'm working at Oracle on the OpenJDK build team and is respons= ible for large parts of the build system of OpenJDK. Lately, I've been play= ing around with FreeBSD (and other BSDs) in my free time, and I've written = a patch that will add build system support for FreeBSD, OpenBSD and NetBSD = in JDK 9 (tracked in >https://bugs.openjdk.java.n= et/browse/JDK-8147795 >). > >>>>>>>>>>=20 > >>>>>>>>>> I started writing this when I realized that the jdk9 branch in= the OpenJDK bsd-port repository did not contain any BSD-specific changes a= t all, and the old JDK 8 changes will not readily be portable, due to major= changes in the build system between JDK 8 and JDK 9. > >>>>>>>>>>=20 > >>>>>>>>>> I thought it would be a no-brainer to integrate these changes = into the JDK 9 mainline, so they would be in place for whenever you guys wo= uld start to attack porting the code base. However, some of my collegues th= ought otherwise. The end result, I think, is that they wanted to see someon= e (not necessarily a company, the FreeBSD organisation for instance seemed = to be okay), to step forward and say "we take responsibility for the BSD po= rt", and give some kind of commitment to actually use these build changes i= n producing a viable port. You can read the mail conversation here: >http:/= /mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html > > >>>>>>>>>=20 > >>>>>>>>> I forgot to add that I'm willing to help with build issues for = the BSD port, as far as my (like everybode else's) limited free time allows= me. Unfortunately, I'm still a noob at BSD internals and can't really help= with much apart from that. > >>>>>>>>>=20 > >>>>>>>>> /Magnus > >>>>>>>>>=20 > >>>>>>>>>=20 > >>>>>>>>>>=20 > >>>>>>>>>> An alternative to pushing this code into the JDK 9 mainline is= of course to push it to the bsd-port/jdk9 repo (given that the owners of t= hat repo approves), but that seems like a less favourable solution. Having = the code in the mainline does not mean that it gets tested automatically, b= ut it means that it will be part of e.g. refactoring, that would otherwise = break a downstream patchset. > >>>>>>>>>>=20 > >>>>>>>>>> I cc:ed this conversation to the >bsd-port-dev@openjdk.jav= a.net > mailing list, but never got= any kind of official response there. Since this list seems more active, I'= m trying here instead. :) > >>>>>>>>>>=20 > >>>>>>>>>> So, I'm offering two patches here, one that applies to the bui= ld system, is nice and clean, and possible to integrate into JDK 9 mainline= , if my collegues are convinced that someone is backing up the BSD port. An= d there's a second patch, which fixes broken C/C++/Java code and results in= a product that can at least run "javac HelloWorld", but this is not yet cl= ean enough for integration anywere, at least not the JDK 9 mainline. (I don= 't know enough of the BSD internals to fix all problems, so there's some "#= if 0" code here and there.) > >>>>>>>>>>=20 > >>>>>>>>>> /Magnus > >>>>>>>>>>=20 > >>>>>>>>>> _______________________________________________ > >>>>>>>>>> freebsd-java@freebsd.org > mailing li= st > >>>>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java > > >>>>>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@fre= ebsd.org " > > >>>>>>>>>=20 > >>>>>>>>> _______________________________________________ > >>>>>>>>> freebsd-java@freebsd.org > >> mailing list > >>>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java > >> > >>>>>>>>> To unsubscribe, send any mail to " >freebsd-java= -unsubscribe@freebsd.org > < >mailto:freebsd-java-unsubscribe@freebsd.org = >>" > >>>>>>>>=20 > >>>>>>>> _______________________________________________ > >>>>>>>> freebsd-java@freebsd.org > mailing list > >>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java > > >>>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freeb= sd.org " > > >>>>>>>=20 > >>>>>>> --=20 > >>>>>>> Greg Lewis Email : >glewis@eyesbeyond.com > > >>>>>>> Eyes Beyond Web : >http://www.eyesbeyond.com > > >>>>>>> Information Technology FreeBSD : >glewis@FreeBSD.org > > >>>>>>> _______________________________________________ > >>>>>>> freebsd-java@freebsd.org > mailing list > >>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java > > >>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freebs= d.org >" > >>>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>=20 > >>>> _______________________________________________ > >>>> freebsd-java@freebsd.org mailing l= ist > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java > >>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freebsd.o= rg " > >>>=20 > >>=20 > >=20 > > --=20 > > Greg Lewis Email : glewis@eyesbeyond.com > > Eyes Beyond Web : http://www.eyesbeyond.com > > Information Technology FreeBSD : glewis@FreeBSD.org >=20 --=20 Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org