Date: Wed, 07 Jan 2015 14:26:00 -0700 From: Sean Bruno <sbruno@ignoranthack.me> To: freebsd-python@freebsd.org Subject: Re: python waf bypasses _MAKE_JOBS number Message-ID: <54ADA468.9070206@ignoranthack.me> In-Reply-To: <1420653060968-5979111.post@n5.nabble.com> References: <54AC74E0.4050404@ignoranthack.me> <20150107072132.GA1050@medusa.sysfault.org> <54AD5BB9.2060701@ignoranthack.me> <1420653060968-5979111.post@n5.nabble.com>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 01/07/15 10:51, sson [via FreeBSD] wrote: > Sean Bruno-6 wrote >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >> >> On 01/07/15 00:21, Marcus von Appen wrote: >>> On, Wed Jan 07, 2015, Sean Bruno wrote: >>> >>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >>>> >>>> Hey, so ... because qemu has a bug in it, we're trying to >>>> enforce no SMP behaviour in builds. >>>> >>>> Turns out that python waf bypasses all of ports logic and >>>> probes for the number of cpus and does its own thing. We >>>> noted this in our builds as they locked up when using qemu >>>> due to a bug. >>>> >>>> Can this behaviour be investigated (configure behaviour) by >>>> some python knowledgeable folks? >>> >>> Can you point us to the waf build logic for the qemu port? >>> Looking at emulators/qemu, I do not see anything of relevance. >>> >>> Cheers Marcus >>> >> >> Should have been a bit clearer, its qemu-bsd-user (via >> qemu-user-static) that I'm talking about. >> >> I've added Stacey to the email to clarify a bit more. >> >> The behavior we are talking about isn't specific to qemu at all, >> its just the configure script will probe the number of cpus and >> ignore my override. >> >> sean _______________________________________________ > >> freebsd-python@ > >> mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-python To >> unsubscribe, send any mail to " > >> freebsd-python-unsubscribe@ > >> " > > Hi all: > > The problem we are seeing is actually described in pretty good > detail in an old a bug report: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=160717 > > It is hard to follow what was actually changed that fixed this > problem on FreeBSD/amd64 but it seems the threading code for > FreeBSD/{arm, mips, mips64} may not be as mature as the amd64 port. > In any case, if the "--jobs=1" flag is used then the ports will > build without hanging. Python waf assumes, however, that the > default jobs number should be the number of cores available on the > system when the "--jobs=#" (or "-j#") is not explicitly given > rather than assuming "--jobs=1". It seems, however, that many (if > not all) the ports that use python waf make the assumption that it > defaults to "--jobs=1". So, in summary: > > (1) It seems that described bug above wasn't fixed for all the > FreeBSD arch's. It is unclear was the cause of the problem was and > what was changed. If it is know what was changed it may be > possible to figure out what needs to be changed to fix this problem > in the other FreeBSD ports. > > (2) It seems to be assumed by many of the port maintainers that > python waf defaults to "--jobs=1" when the argument is not given. > This is problematic for poudriere, for example, given that it > limits the MAKE_JOBS_NUMBER to 1 to better control the load on the > build server. > > Of course, if #2 was fixed then it would solve two problems (for us > anyway). :) > > Best Regards, > > -stacey. > > > I think I can work around python waf assuming it can figure out the number of cpus in the two critical ports tdb/talloc by enforcing a - --jobs=1 https://people.freebsd.org/~sbruno/disable_make_jobs.diff These two are the most obvious in the tree that fail for armv6 qemu builds. sean -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJUraRkXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5kg6kIAMpyFOqqIxhbsDI1HBFAWIM6 j/ci3S1i7H0uVzoTmvGn3ITDY8zAPG3cDm1EaPPcn5vUNvAnk9uzo0ip33OZ22WM 8mJza2vGyoWLFGdrZPwm/+izWdXDoGqLX1ZSvzQlT8elyD2DlzduX++UJNdLgBkU assRj4HUFxe/1OQO911bZXfQVrsU9XbvssDrCvgZlQGo7siEDrQUpbQgqfDviQjK SXxJy8qM7l60309w13TmZpCriE3nTmNMdUazTV5mXS/NhQqkL5ckiv0uHGrn6/a7 KC/L+whcjuT4FF86n0qCyrBiSsEemxOywIIdVd+r2X75r0pmXCw6214CI6QuZ40= =f2RT -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54ADA468.9070206>