Date: Sat, 13 Feb 2016 14:56:35 -0800 From: Greg Lewis <glewis@eyesbeyond.com> To: Brian Gardner <openjdk@getsnappy.com> Cc: Magnus Ihse Bursie <magnus.ihse.bursie@oracle.com>, java@freebsd.org Subject: Re: Helping out with JDK 9 on BSD Message-ID: <20160213225635.GB64267@misty.eyesbeyond.com> In-Reply-To: <99C45649-B498-4BE1-97D4-0CEE0BC8A77D@getsnappy.com> References: <56ADE943.6020103@oracle.com> <8B9ACABA-F9B2-4288-9DB5-CA843DE48C34@getsnappy.com> <20160203034053.GA36170@misty.eyesbeyond.com> <A4624A53-1329-481E-928C-F2C8167BA001@getsnappy.com> <56B45453.9020006@oracle.com> <A0BD54C7-2DAE-4B7A-B28C-B639E138DA56@getsnappy.com> <1C39812E-E596-47E6-AB45-0A591A017FFB@getsnappy.com> <F993524B-0332-4B9C-93CE-FA3D2071F29F@getsnappy.com> <20160213080945.GC56357@misty.eyesbeyond.com> <99C45649-B498-4BE1-97D4-0CEE0BC8A77D@getsnappy.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I'm probably going to look at these piecemeal... On Sat, Feb 13, 2016 at 11:08:36AM -0800, Brian Gardner wrote: > I???m sending those links again, it came through weird on the mailing list > porting build_vm_def.sh from bsd-port/jdk8 repo, to fix NM errors during = build > http://brian.timestudybuddy.com/webrev/hotspot__NM/webrev/ This one looks straight forward. +1 > Add SUPPORT_RESERVED_STACK_AREA flag for all BSD's > http://brian.timestudybuddy.com/webrev/hotspot__SUPPORT_RESERVED_STACK_AR= EA/webrev/ > porting getthreadid logic from bsd-port/jdk8. calling syscall(SYS_thr_sel= f) caused pthread_setspecific to be cleared. > http://brian.timestudybuddy.com/webrev/hotspot__os_bsd_cpp__getthreadid/w= ebrev/ > adding in servicability agent ported from bsd-port/jdk8 > http://brian.timestudybuddy.com/webrev/hotspot__sa/webrev/ > adding classlist.bsd that is identical to classlist.linux, in order to co= mpile > http://brian.timestudybuddy.com/webrev/jdk__classlist-bsd/webrev/ This also looks straight forward. There are differences in openjdk8, but frankly the differences just make it look like they are out of sync, not that they meaningfully differ. +1 > clean up TLS current thread at end of ::run functions similar to how it's= done in openjdk8. > http://brian.timestudybuddy.com/webrev/hotspot__clear_thread_current/webr= ev/ > clear current thread before exiting java_start to avoid warnings from lef= tover pthread_setspecific data > http://brian.timestudybuddy.com/webrev/hotspot__clear_thread_current_alt/= webrev/ Will look at the others as I have time. >=20 > > On Feb 13, 2016, at 12:09 AM, Greg Lewis <glewis@eyesbeyond.com> wrote: > >=20 > > On Fri, Feb 12, 2016 at 10:54:29AM -0800, Brian Gardner wrote: > >> After giving it some thought, I???ve created an alternate patch for th= e clear_current_thread issue. I like this solution better because it is is= olated to java_start function in os_bsd.cpp, and it makes java_start respon= sible for cleaning up the current_thread since it also initializes it. > >=20 > > Sounds good! What can I do to help get all the changes up for review? > >=20 > > -- Greg > >=20 > >>> On Feb 10, 2016, at 10:45 PM, Brian Gardner <openjdk@getsnappy.com> w= rote: > >>>=20 > >>> Hello again Greg and Magnus, > >>> I finished up researching the leftover thread-specific data after 4 d= estructor iterations issue. It looks like it???s currently functioning on = Freebsd the same way it does on Linux, the only difference is that Freebsd = output that warning were Linux silently stops after it???s forth try. I lo= oked into how openjdk8 was working and it looks like there were calls to TL= S::set_thread(NULL) scattered throughout the code base at the end of some := :run methods. I???ve mimicked that behavior in the necessary spots with th= e following patch. I can???t help but think there must be a better way to = get these Thread???s destructors called, and I tried that in place of makin= g clear_thread_current public and calling it directly, but it resulted in d= eadlock. > >>>=20 > >>> <hotspot__clear_thread_current.patch> > >>>=20 > >>>> On Feb 9, 2016, at 2:36 PM, Brian Gardner <openjdk@getsnappy.com <ma= ilto:openjdk@getsnappy.com>> wrote: > >>>>=20 > >>>> Hi Greg and Magnus, > >>>> I???ve attached the patches needed to build and compile on Freebsd. = I???ve reverted my changes to TLS, and done more debugging there. There a= ppears to be two separate issues. One issue was that the implementation of= OS::bad::gettid() was somehow clearing the current thread previously set w= ith pthread_setspecific. I was able to fix this by porting logic from bad-= port/jdk8, see hotspot__os_bd_cpp__getthreadid.patch. The second issue see= ms to be related to certain types of threads never getting their destructor= called and therefor not clearing pthread_setspecific and resulting in Thre= ad 803c59000 has exited with leftover thread-specific data after 4 destruct= or iterations, errors. This one I haven???t tracked down yet and will cont= inue looking into. While I can compile and execute HelloWorld, I???m getti= ng a ton of these benign yet annoying messages from ConcurrentGC and Worker= threads. I???ll continue looking into that next week. > >>>>=20 > >>>>=20 > >>>>> On Feb 4, 2016, at 11:50 PM, Magnus Ihse Bursie <magnus.ihse.bursie= @oracle.com <mailto:magnus.ihse.bursie@oracle.com>> wrote: > >>>>>=20 > >>>>> On 2016-02-04 01:29, Brian Gardner wrote: > >>>>>> Hi Greg, > >>>>>> Great to hear from you, it???s been a while since I took on one of= these projects, but I do follow the java@freebsd mailing list and see all = the work you put in. I must tell you how greatly your work is appreciated b= y me and I???m sure the rest of the community. I have good news, with fair= ly minimal changes I have it compiling, building HelloWorld, and running He= lloWorld all without errors. > >>>>>>=20 > >>>>>> I think it???ll break down into 4 change sets, > >>>>>> NM - I ported some code from bsd-port/jdk8/hotspot/make/bsd/makefi= les/build_vm_def.sh that fixes a compile error=20 > >>>>>> SUPPORT_RESERVED_STACK_AREA - define SUPPORT_RESERVED_STACK_AREA f= or _ALLBSD_SOURCE > >>>>>> Serviceability Agent - ported from bad-port/jdk8, currently compil= es but hasn???t been tested > >>>>>> TLS - there is problems with the TLS in jdk9 used for Thread::curr= ent(). This caused several issues for me and while the fix was relatively = straight forward in using THREAD_LOCAL_DECL completely in place of TLS comp= letely, I???d imagine there was a reason it was implemented in sort of a Hy= brid fashion alongside TLS. > >>>>>=20 > >>>>> There is a reason. It is very well described in this comment: > >>>>> https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommentId= =3D13866505 <https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommen= tId=3D13866505> <https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCo= mmentId=3D13866505 <https://bugs.openjdk.java.net/browse/JDK-8132510?focuse= dCommentId=3D13866505>> > >>>>>=20 > >>>>> The TL;DR: in some circumstances the JVM can crash if we do not set= up an initial TLS. On the other hand, this was for glibc, so maybe this si= tuation does not arise for FreeBSD. > >>>>>=20 > >>>>>>=20 > >>>>>> It would be great if I could post these as web reviews for others = to review. Greg, would getting access to bsd-port/jdk9 allow us to collabo= rate through web reviews? =20 > >>>>>=20 > >>>>>=20 > >>>>>> Magnus, do you think using bsd-port/jdk9 for this purpose be a ste= p towards getting these basic changes into the jdk9 mainline, or would your= colleagues think "if they have bsd-port/jdk9, why bother with integrating = them into the jdk9 mainline???? > >>>>>=20 > >>>>> I think putting these changes in bsd-port/jdk9 is very reasonable, = and I do not believe it will in any way hinder their acceptance into jdk9 m= ainline. The only thing to be careful about there, though, is that all cont= ributors have signed the OCA (Oracle contributor agreement). Mixing in "leg= ally bad" code can present a real problem for adoption into mainline. > >>>>>=20 > >>>>> /Magnus > >>>>>=20 > >>>>>>=20 > >>>>>> Brian Gardner > >>>>>>=20 > >>>>>>> On Feb 2, 2016, at 7:40 PM, Greg Lewis < <mailto:glewis@eyesbeyon= d.com <mailto:glewis@eyesbeyond.com>>glewis@eyesbeyond.com <mailto:glewis@e= yesbeyond.com> <mailto:glewis@eyesbeyond.com <mailto:glewis@eyesbeyond.com>= >> wrote: > >>>>>>>=20 > >>>>>>> On Sun, Jan 31, 2016 at 06:57:26AM -0800, Brian Gardner wrote: > >>>>>>>> I???m interested in helping. I ported openjdk6 to freebsd and a= lso helped out with openjdk8. > >>>>>>>>=20 > >>>>>>>> Brian Gardner > >>>>>>>=20 > >>>>>>> I'd love to see as much of these changes get into the jdk9 mainli= ne. My time > >>>>>>> is really limited for the next couple of weeks though. If it wou= ld help you > >>>>>>> or Magnus to use the bsd-port repo to stage changes then that wou= ld be > >>>>>>> great too. Let me do a merge to make sure it is up to date with = mainline. > >>>>>>> Notionally we'd have to do a vote to get you both access I think?= But I > >>>>>>> can shepherd changes in if that will help with sharing them. > >>>>>>>=20 > >>>>>>> FWIW, in terms of taking responsibility, I've been regularly upda= ting the > >>>>>>> repos for jdk8 and jdk7 for a couple of years. I'm happy to keep= jdk9 > >>>>>>> building if that is what is being looked for there and also port = more of > >>>>>>> the BSD changes for jdk8 as appropriate. > >>>>>>>=20 > >>>>>>> - Greg > >>>>>>>=20 > >>>>>>>>> On Jan 31, 2016, at 3:00 AM, Magnus Ihse Bursie <magnus.ihse.bu= rsie@oracle.com <mailto:magnus.ihse.bursie@oracle.com> <mailto:magnus.ihse.= bursie@oracle.com <mailto:magnus.ihse.bursie@oracle.com>>> wrote: > >>>>>>>>>=20 > >>>>>>>>> On 2016-01-31 11:54, Magnus Ihse Bursie wrote: > >>>>>>>>>> Hi, > >>>>>>>>>>=20 > >>>>>>>>>> [TL;DR: I'm offering patches to compile JDK 9 on FreeBSD, but = it's not clear where to put them, or how.] > >>>>>>>>>>=20 > >>>>>>>>>> I'm working at Oracle on the OpenJDK build team and is respons= ible for large parts of the build system of OpenJDK. Lately, I've been play= ing around with FreeBSD (and other BSDs) in my free time, and I've written = a patch that will add build system support for FreeBSD, OpenBSD and NetBSD = in JDK 9 (tracked in <https://bugs.openjdk.java.net/browse/JDK-8147795 <ht= tps://bugs.openjdk.java.net/browse/JDK-8147795>>https://bugs.openjdk.java.n= et/browse/JDK-8147795 <https://bugs.openjdk.java.net/browse/JDK-8147795><ht= tps://bugs.openjdk.java.net/browse/JDK-8147795 <https://bugs.openjdk.java.n= et/browse/JDK-8147795>>). > >>>>>>>>>>=20 > >>>>>>>>>> I started writing this when I realized that the jdk9 branch in= the OpenJDK bsd-port repository did not contain any BSD-specific changes a= t all, and the old JDK 8 changes will not readily be portable, due to major= changes in the build system between JDK 8 and JDK 9. > >>>>>>>>>>=20 > >>>>>>>>>> I thought it would be a no-brainer to integrate these changes = into the JDK 9 mainline, so they would be in place for whenever you guys wo= uld start to attack porting the code base. However, some of my collegues th= ought otherwise. The end result, I think, is that they wanted to see someon= e (not necessarily a company, the FreeBSD organisation for instance seemed = to be okay), to step forward and say "we take responsibility for the BSD po= rt", and give some kind of commitment to actually use these build changes i= n producing a viable port. You can read the mail conversation here: <http:/= /mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html <http:/= /mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html>>http:/= /mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html <http:/= /mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html> <http:= //mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html <http:= //mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html>> > >>>>>>>>>=20 > >>>>>>>>> I forgot to add that I'm willing to help with build issues for = the BSD port, as far as my (like everybode else's) limited free time allows= me. Unfortunately, I'm still a noob at BSD internals and can't really help= with much apart from that. > >>>>>>>>>=20 > >>>>>>>>> /Magnus > >>>>>>>>>=20 > >>>>>>>>>=20 > >>>>>>>>>>=20 > >>>>>>>>>> An alternative to pushing this code into the JDK 9 mainline is= of course to push it to the bsd-port/jdk9 repo (given that the owners of t= hat repo approves), but that seems like a less favourable solution. Having = the code in the mainline does not mean that it gets tested automatically, b= ut it means that it will be part of e.g. refactoring, that would otherwise = break a downstream patchset. > >>>>>>>>>>=20 > >>>>>>>>>> I cc:ed this conversation to the <mailto:bsd-port-dev@openjdk= =2Ejava.net <mailto:bsd-port-dev@openjdk.java.net>>bsd-port-dev@openjdk.jav= a.net <mailto:bsd-port-dev@openjdk.java.net> <mailto:bsd-port-dev@openjdk.j= ava.net <mailto:bsd-port-dev@openjdk.java.net>> mailing list, but never got= any kind of official response there. Since this list seems more active, I'= m trying here instead. :) > >>>>>>>>>>=20 > >>>>>>>>>> So, I'm offering two patches here, one that applies to the bui= ld system, is nice and clean, and possible to integrate into JDK 9 mainline= , if my collegues are convinced that someone is backing up the BSD port. An= d there's a second patch, which fixes broken C/C++/Java code and results in= a product that can at least run "javac HelloWorld", but this is not yet cl= ean enough for integration anywere, at least not the JDK 9 mainline. (I don= 't know enough of the BSD internals to fix all problems, so there's some "#= if 0" code here and there.) > >>>>>>>>>>=20 > >>>>>>>>>> /Magnus > >>>>>>>>>>=20 > >>>>>>>>>> _______________________________________________ > >>>>>>>>>> freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> <ma= ilto:freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org>> mailing li= st > >>>>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https= ://lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.= org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listin= fo/freebsd-java>> > >>>>>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@fre= ebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org>" <mailto:freebsd-jav= a-unsubscribe@freebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org>> > >>>>>>>>>=20 > >>>>>>>>> _______________________________________________ > >>>>>>>>> freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> <mai= lto:freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org>> <mailto:fre= ebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> <mailto:freebsd-jav= a@freebsd.org <mailto:freebsd-java@freebsd.org>>> mailing list > >>>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https:= //lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.o= rg/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinf= o/freebsd-java>> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <= https://lists.freebsd.org/mailman/listinfo/freebsd-java>; <https://lists.fre= ebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/l= istinfo/freebsd-java>>> > >>>>>>>>> To unsubscribe, send any mail to " <mailto:freebsd-java-unsubsc= ribe@freebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org>>freebsd-java= -unsubscribe@freebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org> <mai= lto:freebsd-java-unsubscribe@freebsd.org <mailto:freebsd-java-unsubscribe@f= reebsd.org>> < <mailto:freebsd-java-unsubscribe@freebsd.org <mailto:freebsd= -java-unsubscribe@freebsd.org>>mailto:freebsd-java-unsubscribe@freebsd.org = <mailto:freebsd-java-unsubscribe@freebsd.org> <mailto:freebsd-java-unsubscr= ibe@freebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org>>>" > >>>>>>>>=20 > >>>>>>>> _______________________________________________ > >>>>>>>> freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> <mail= to:freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org>> mailing list > >>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https:/= /lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.or= g/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo= /freebsd-java>> > >>>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freeb= sd.org <mailto:freebsd-java-unsubscribe@freebsd.org>" <mailto:freebsd-java-= unsubscribe@freebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org>> > >>>>>>>=20 > >>>>>>> --=20 > >>>>>>> Greg Lewis Email : <mailto:glewis@eye= sbeyond.com <mailto:glewis@eyesbeyond.com>>glewis@eyesbeyond.com <mailto:gl= ewis@eyesbeyond.com> <mailto:glewis@eyesbeyond.com <mailto:glewis@eyesbeyon= d.com>> > >>>>>>> Eyes Beyond Web : <http://www.eyesbe= yond.com/ <http://www.eyesbeyond.com/>>http://www.eyesbeyond.com <http://ww= w.eyesbeyond.com/> <http://www.eyesbeyond.com/ <http://www.eyesbeyond.com/>>; > >>>>>>> Information Technology FreeBSD : <mailto:glewis@fre= ebsd.org <mailto:glewis@freebsd.org>>glewis@FreeBSD.org <mailto:glewis@free= bsd.org> <mailto:glewis@FreeBSD.org <mailto:glewis@FreeBSD.org>> > >>>>>>> _______________________________________________ > >>>>>>> freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> <mailt= o:freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org>> mailing list > >>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://= lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.org= /mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/= freebsd-java>> > >>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freebs= d.org <mailto:freebsd-java-unsubscribe@freebsd.org> <mailto:freebsd-java-un= subscribe@freebsd.org <mailto:freebsd-java-unsubscribe@freebsd.org>>" > >>>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>=20 > >>>> _______________________________________________ > >>>> freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> mailing l= ist > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lis= ts.freebsd.org/mailman/listinfo/freebsd-java> > >>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freebsd.o= rg <mailto:freebsd-java-unsubscribe@freebsd.org>" > >>>=20 > >>=20 > >=20 > > --=20 > > Greg Lewis Email : glewis@eyesbeyond.com > > Eyes Beyond Web : http://www.eyesbeyond.com > > Information Technology FreeBSD : glewis@FreeBSD.org >=20 --=20 Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160213225635.GB64267>