Date: Sat, 12 May 2007 20:11:49 -0500 From: Stephen Montgomery-Smith <stephen@math.missouri.edu> To: "[LoN]Kamikaze" <LoN_Kamikaze@gmx.de> Cc: freebsd-ports@freebsd.org, Kris Kennaway <kris@obsecurity.org> Subject: Re: Time to abandon recursive pulling of dependencies? Message-ID: <464665D5.1090509@math.missouri.edu> In-Reply-To: <4646193E.5040503@gmx.de> References: <464597C6.3030406@gmx.de> <20070512174011.GA22526@xor.obsecurity.org> <4645FF71.60100@gmx.de> <20070512175824.GA23103@xor.obsecurity.org> <20070512133054.B5588@math.missouri.edu> <4646193E.5040503@gmx.de>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------080702080209080907050903 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit OK chaps, this is what I came up with. So for example, if I do "make install" on /usr/ports/x11/xorg (having made all the dependencies), on my computer it turns the pkg_create from taking about 4 minutes to the blink of an eye. Now people need to figure out how to speed up the "make package-depends" in bsd.ports.mk, but that is beyond my abilities. I really hope this works. The prospect of modifying a piece of code that is used by practically the whole FreeBSD community kind of scares me, so I would appreciate some good testing. Apply the patch http://www.math.missouri.edu/~stephen/deps/ddd to /usr/src/usr.sbin/pkg_install/lib. I have also put the patch as an attachment, but I don't know if the mail filters will take it out. Stephen --------------080702080209080907050903 Content-Type: text/plain; name="ddd" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="ddd" LS0tIGRlcHMuYy1vcmlnCVNhdCBNYXkgMTIgMTk6MDI6MjEgMjAwNworKysgZGVwcy5jCVNh dCBNYXkgMTIgMTk6NTY6MTcgMjAwNwpAQCAtMjYsOTggKzI2LDEwNSBAQAogI2luY2x1ZGUg PGVyci5oPgogI2luY2x1ZGUgPHN0ZGlvLmg+CiAKK3ZvaWQgbGlzdF9kZXBzKGNvbnN0IGNo YXIgKnBrZ25hbWUsIGNoYXIgKipwa2dzLCBjaGFyICpsaXN0ZWQsIAorICAgICAgICAgICAg ICAgY2hhciAqY2hlY2tfbG9vcCwgY2hhciAqKm5ld3BrZ3MsIGludCAqbnJuZXdwa2dzLCBp bnQgKmVycmNvdW50KTsKKwogLyoKICAqIFNvcnQgZ2l2ZW4gTlVMTC10ZXJtaW5hdGVkIGxp c3Qgb2YgaW5zdGFsbGVkIHBhY2thZ2VzIChwa2dzKSBpbgogICogc3VjaCBhIHdheSB0aGF0 IGlmIHBhY2thZ2UgQSBkZXBlbmRzIG9uIHBhY2thZ2UgQiB0aGVuIGFmdGVyCiAgKiBzb3J0 aW5nIEEgd2lsbCBiZSBsaXN0ZWQgYmVmb3JlIEIgbm8gbWF0dGVyIGhvdyB0aGV5IHdlcmUK ICAqIG9yaWdpbmFsbHkgcG9zaXRpb25lZCBpbiB0aGUgbGlzdC4KKyAqCisgKiBXb3JrcyBi eSBwZXJmb3JtaW5nIGEgcmVjdXJzaXZlIGRlcHRoLWZpcnN0IHNlYXJjaCBvbiB0aGUgcmVx dWlyZWQtYnkgbGlzdHMuCiAgKi8KKwogaW50CiBzb3J0ZGVwcyhjaGFyICoqcGtncykKIHsK LSAgICBjaGFyICp0bXA7Ci0gICAgaW50IGksIGosIGxvb3BfY250OwotICAgIGludCBlcnJf Y250ID0gMDsKKyAgICBpbnQgaSwgZXJyY291bnQ9MDsKKyAgICBpbnQgbnJwa2dzLCBucm5l d3BrZ3M7CisgICAgY2hhciAqbGlzdGVkLCAqY2hlY2tfbG9vcCwgKipuZXdwa2dzOworICAg IGNoYXIgKmNwOwogCiAgICAgaWYgKHBrZ3NbMF0gPT0gTlVMTCB8fCBwa2dzWzFdID09IE5V TEwpCiAJcmV0dXJuICgwKTsKIAotICAgIGZvciAoaSA9IDA7IHBrZ3NbaSArIDFdOyBpKysp IHsKLQkvKgotCSAqIENoZWNrIHRvIHNlZSBpZiBhbnkgb3RoZXIgcGFja2FnZSBpbiBwa2dz W2krMTpdIGRlcGVuZHMKLQkgKiBvbiBwa2dzW2ldIGFuZCBzd2FwIHRob3NlIHR3byBwYWNr YWdlcyBpZiBzby4KLQkgKi8KLQlsb29wX2NudCA9IDA7Ci0JZm9yIChqID0gaSArIDE7IHBr Z3Nbal07IGorKykgewotCSAgICBpZiAoY2hraWZkZXBlbmRzKHBrZ3Nbal0sIHBrZ3NbaV0p ID09IDEpIHsKLQkJLyoKLQkJICogVHJ5IHRvIGF2b2lkIGRlYWRsb2NrIGlmIHBhY2thZ2Ug QSBkZXBlbmRzIG9uIEIgd2hpY2ggaW4KLQkJICogdHVybiBkZXBlbmRzIG9uIEMgYW5kIEMg ZHVlIHRvIGFuIGVycm9yIGRlcGVuZHMgb24gQS4KLQkJICogVXNlIHVnbHkgYnV0IHNpbXBs ZSBtZXRob2QsIGJlY2FzZSBpdCBTaG91bGQgTmV2ZXIKLQkJICogSGFwcGVuW3RtXSBpbiB0 aGUgcmVhbCBsaWZlIGFueXdheS4KLQkJICovCi0JCWlmIChsb29wX2NudCA+IDQwOTYpIHsK LQkJICAgIHdhcm54KCJkZXBlbmRlbmN5IGxvb3AgZGV0ZWN0ZWQgZm9yIHBhY2thZ2UgJXMi LCBwa2dzW2pdKTsKLQkJICAgIGVycl9jbnQrKzsKLQkJICAgIGJyZWFrOwotCQl9Ci0JCWxv b3BfY250Kys7Ci0JCXRtcCA9IHBrZ3NbaV07Ci0JCXBrZ3NbaV0gPSBwa2dzW2pdOwotCQlw a2dzW2pdID0gdG1wOwotCQkvKgotCQkgKiBBbm90aGVyIGl0ZXJhdGlvbiByZXF1cmVkIHRv IGNoZWNrIGlmIG5ldyBwa2dzW2ldCi0JCSAqIGl0c2VsZiBoYXMgYW55IHBhY2thZ2VzIHRo YXQgZGVwZW5kIG9uIGl0Ci0JCSAqLwotCQlqID0gaSArIDE7Ci0JICAgIH0KLQl9CisgICAg bnJwa2dzID0gMDsKKyAgICB3aGlsZSAocGtnc1tucnBrZ3NdKSBucnBrZ3MrKzsKKyAgICBs aXN0ZWQgPSBtYWxsb2MobnJwa2dzKTsKKyAgICBiemVybyhsaXN0ZWQsbnJwa2dzKTsKKyAg ICBjaGVja19sb29wID0gbWFsbG9jKG5ycGtncyk7CisgICAgYnplcm8oY2hlY2tfbG9vcCxu cnBrZ3MpOworICAgIG5ld3BrZ3MgPSBtYWxsb2MobnJwa2dzKnNpemVvZihjaGFyKikpOwor ICAgIG5ybmV3cGtncyA9IDA7CisKKyAgICBmb3IgKGkgPSAwOyBwa2dzW2ldOyBpKyspIGlm ICghbGlzdGVkW2ldKSB7CisJY2hlY2tfbG9vcFtpXSA9IDE7CisJY3AgPSBzdHJjaHIocGtn c1tpXSwgJzonKTsKKwlpZiAoY3AgIT0gTlVMTCkKKwkgICAgKmNwID0gJ1wwJzsKKwlsaXN0 X2RlcHMocGtnc1tpXSxwa2dzLGxpc3RlZCxjaGVja19sb29wLG5ld3BrZ3MsJm5ybmV3cGtn cywmZXJyY291bnQpOworCWlmIChjcCAhPSBOVUxMKQorCSAgICAqY3AgPSAnOic7CisJbGlz dGVkW2ldID0gMTsKKwluZXdwa2dzW25ybmV3cGtnc10gPSBwa2dzW2ldOworCW5ybmV3cGtn cysrOwogICAgIH0KLSAgICByZXR1cm4gZXJyX2NudDsKKworICAgIGlmIChucm5ld3BrZ3Mg IT0gbnJwa2dzKSB7CisJZnByaW50ZihzdGRlcnIsIkh1Z2UgZXJyb3IgaW4gY29kZVxuIik7 CisJZXhpdCgxKTsKKyAgICB9CisgICAgZm9yIChpID0gMDsgaSA8IG5ybmV3cGtnczsgaSsr KSBwa2dzW2ldID0gbmV3cGtnc1tpXTsKKworICAgIHJldHVybiBlcnJjb3VudDsKIH0KIAog LyoKLSAqIENoZWNrIHRvIHNlZSBpZiBwa2duYW1lMSBkZXBlbmRzIG9uIHBrZ25hbWUyLgot ICogUmV0dXJucyAxIGlmIGRlcGVuZHMsIDAgaWYgbm90LCBhbmQgLTEgaWYgZXJyb3Igb2Nj dXJlZC4KLSAqLyAKLWludAotY2hraWZkZXBlbmRzKGNvbnN0IGNoYXIgKnBrZ25hbWUxLCBj b25zdCBjaGFyICpwa2duYW1lMikKLXsKLSAgICBjaGFyICpjcDEsICpjcDI7Ci0gICAgaW50 IGVycmNvZGU7CisgKiBUaGlzIHJlY3Vyc2l2ZSBmdW5jdGlvbiBsaXN0cyB0aGUgZGVwZW5k ZW5jaWVzICh0aGF0IGlzLCB0aGUgInJlcXVpcmVkLWJ5InMpCisgKiBmb3IgcGtnbmFtZSwg cHV0dGluZyB0aGVtIGludG8gbmV3cGtncy4KKyAqLworCit2b2lkIGxpc3RfZGVwcyhjb25z dCBjaGFyICpwa2duYW1lLCBjaGFyICoqcGtncywgY2hhciAqbGlzdGVkLCAKKyAgICAgICAg ICAgICAgIGNoYXIgKmNoZWNrX2xvb3AsIGNoYXIgKipuZXdwa2dzLCBpbnQgKm5ybmV3cGtn cywgaW50ICplcnJjb3VudCkgeworICAgIGNoYXIgKmNwOworICAgIGludCBlcnJjb2RlLCBq OwogICAgIHN0cnVjdCByZXFyX2J5X2VudHJ5ICpyYl9lbnRyeTsKICAgICBzdHJ1Y3QgcmVx cl9ieV9oZWFkICpyYl9saXN0OwogCi0gICAgY3AyID0gc3RyY2hyKHBrZ25hbWUyLCAnOicp OwotICAgIGlmIChjcDIgIT0gTlVMTCkKLQkqY3AyID0gJ1wwJzsKLSAgICBjcDEgPSBzdHJj aHIocGtnbmFtZTEsICc6Jyk7Ci0gICAgaWYgKGNwMSAhPSBOVUxMKQotCSpjcDEgPSAnXDAn OwotCi0gICAgZXJyY29kZSA9IDA7Ci0gICAgLyogQ2hlY2sgdGhhdCBwa2duYW1lMiBpcyBh Y3R1YWxseSBpbnN0YWxsZWQgKi8KLSAgICBpZiAoaXNpbnN0YWxsZWRwa2cocGtnbmFtZTIp IDw9IDApCi0JZ290byBleGl0OworICAgIGlmIChpc2luc3RhbGxlZHBrZyhwa2duYW1lKSA8 PSAwKQorCXJldHVybjsKIAotICAgIGVycmNvZGUgPSByZXF1aXJlZGJ5KHBrZ25hbWUyLCAm cmJfbGlzdCwgRkFMU0UsIFRSVUUpOworICAgIGVycmNvZGUgPSByZXF1aXJlZGJ5KHBrZ25h bWUsICZyYl9saXN0LCBGQUxTRSwgVFJVRSk7CiAgICAgaWYgKGVycmNvZGUgPCAwKQotCWdv dG8gZXhpdDsKKwlyZXR1cm47CiAKLSAgICBlcnJjb2RlID0gMDsKLSAgICBTVEFJTFFfRk9S RUFDSChyYl9lbnRyeSwgcmJfbGlzdCwgbGluaykgewotCWlmIChzdHJjbXAocmJfZW50cnkt PnBrZ25hbWUsIHBrZ25hbWUxKSA9PSAwKSB7CS8qIG1hdGNoICovCi0JICAgIGVycmNvZGUg PSAxOwotCSAgICBicmVhazsKKyAgICBTVEFJTFFfRk9SRUFDSChyYl9lbnRyeSwgcmJfbGlz dCwgbGluaykKKwlmb3IgKGogPSAwOyBwa2dzW2pdOyBqKyspIGlmICghbGlzdGVkW2pdKSB7 CisJICAgIGNwID0gc3RyY2hyKHBrZ3Nbal0sICc6Jyk7CisJICAgIGlmIChjcCAhPSBOVUxM KQorCQkqY3AgPSAnXDAnOworCSAgICBpZiAoc3RyY21wKHJiX2VudHJ5LT5wa2duYW1lLCBw a2dzW2pdKSA9PSAwKSB7IC8qbWF0Y2ggKi8KKwkJLyoKKwkJICogVHJ5IHRvIGF2b2lkIGRl YWRsb2NrIGlmIHBhY2thZ2UgQSBkZXBlbmRzIG9uIEIgd2hpY2ggaW4KKwkJICogdHVybiBk ZXBlbmRzIG9uIEMgYW5kIEMgZHVlIHRvIGFuIGVycm9yIGRlcGVuZHMgb24gQS4KKwkJICog SXQgU2hvdWxkIE5ldmVyIEhhcHBlblt0bV0gaW4gdGhlIHJlYWwgbGlmZS4KKwkJICovCisJ CWlmIChjaGVja19sb29wW2pdKSB7CisJCSAgICB3YXJueCgiZGVwZW5kZW5jeSBsb29wIGRl dGVjdGVkIGZvciBwYWNrYWdlICVzIiwgcGtnc1tqXSk7CisJCSAgICAoKmVycmNvdW50KSsr OworCQl9CisJCWVsc2UgeworCQkgICAgY2hlY2tfbG9vcFtqXSA9IDE7CisJCSAgICBsaXN0 X2RlcHMocGtnc1tqXSxwa2dzLGxpc3RlZCxjaGVja19sb29wLG5ld3BrZ3MsbnJuZXdwa2dz LGVycmNvdW50KTsKKwkJICAgIGxpc3RlZFtqXSA9IDE7CisJCSAgICBuZXdwa2dzWypucm5l d3BrZ3NdID0gcGtnc1tqXTsKKwkJICAgICgqbnJuZXdwa2dzKSsrOworCQl9CisJICAgIH0K KwkgICAgaWYgKGNwICE9IE5VTEwpCisJCSpjcCA9ICc6JzsKIAl9Ci0gICAgfQotCi1leGl0 OgotICAgIGlmIChjcDEgIT0gTlVMTCkKLQkqY3AxID0gJzonOwotICAgIGlmIChjcDIgIT0g TlVMTCkKLQkqY3AyID0gJzonOwotICAgIHJldHVybiBlcnJjb2RlOwogfQogCiAvKgo= --------------080702080209080907050903--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?464665D5.1090509>