Date: Thu, 29 Sep 2005 13:06:26 -0400 (EDT) From: Rob Watt <rob@hudson-trading.com> To: Robert Watson <rwatson@FreeBSD.org> Cc: Rob Watt <rob@hudson-trading.com>, mikep@hudson-trading.com, freebsd-hackers@FreeBSD.org, freebsd-amd64@FreeBSD.org, Jason Carroll <jason@hudson-trading.com> Subject: Re: freebsd-5.4-stable panics Message-ID: <20050929124027.U65402@daemon.mistermishap.net> In-Reply-To: <20050927222624.R34322@fledge.watson.org> References: <da4a53d805092310237d732554@mail.gmail.com> <20050925115912.H11229@fledge.watson.org> <20050927140535.G50334@daemon.mistermishap.net> <20050927203128.S61419@fledge.watson.org> <cf6c78405092714227722d534@mail.gmail.com> <20050927222624.R34322@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --0-432868459-1128013586=:65402 Content-Type: TEXT/PLAIN; charset=US-ASCII Robert, We have gotten some more information from our type1 crash: >sh lockedvnods Locked vnodes >sh alllocks Process 2204 (dataplay) thread 0xffffff00b1726a000 (100214) exclusive sleep mutex inp (udpinp) f = 0 (0xffffff00cc90fcc8) locked @ /usr/src/sys/netinet/udp_usrreq.c:762 Process 62 (pagedaemon) thread 0xffffff00e358c280 (100049) exclusive sleep mutex UMA lock r = 0 (0xffffffff8068bf80) locked @ /usr/src/sys/vm/uma_core.c:1491 exclusive sleep mutex Giant r = 0 (0xffffffff8062ed80) locked @ /usr/src/sys/vm/vm_pageout.c:717 Process 48 (swi1:net) thread 0xffffff00e3597780 (100027) exclusive sleep mutex IPFW static rules r = 0 (0xffffffff8067ae50) locked @ /usr/src/sys/netinet/ip_fw2.c:149 >sh pcpu cpuid=0 currthread = 0xffffff00e358c280: pid 63 "pagedaemon" currpcb = 0xffffffffb34e3d10 fpcurrthread = none idle thread = 0xffffff00e35b6000: pid 14 (idle cpu0) spin locks held = >sh pcpu 1 cpuid=1 currthread = 0xffffff00e358b3c80: pid 13 "idle cpu1" currpcb = 0xffffffffffb34e7d10 fpcurrthread = none idle thread = 0xffffff00e358b3c80: pid 13 (idle cpu1) spin locks held = >sh pcpu 2 cpuid=2 currthread = 0xffffff00e35e4000: pid 2715 "bonnie" currpcb = 0xffffffffffb636dd10 fpcurrthread = none idle thread = 0xffffff00e35b3a00: pid 12 (idle cpu2) spin locks held = >sh pcpu 3 cpuid=3 currthread = 0xffffff00e35aea00: pid 40 "irq27: em1 em2" currpcb = 0xffffffffffb34b6d10 fpcurrthread = none idle thread = 0xffffff00e35b3780: pid 11 (idle cpu0) spin locks held = I have attached the core output as type1-core.2.txt, but unfortunately it does not help us determine the area of code that triggered the exception. If I can get more DDB output from the type2 crash I will post it. There is some encouraging news: since we stopped running top, our 6.0-BETA5 test machine has not crashed (its been running tests now for over 26 hours). We also started running tests on a dual single-core machine running 5-STABLE. That machine has been running for 50 hours without crashing. This means that we are now only hitting these bugs with dual dual-core machines running 5-STABLE. - Rob Watt --0-432868459-1128013586=:65402 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="type1-core.2.txt" Content-Transfer-Encoding: BASE64 Content-Description: Content-Disposition: attachment; filename="type1-core.2.txt" RERCOg0KPnNoIGxvY2tlZHZub2RzDQpMb2NrZWQgdm5vZGVzDQoNCj5zaCBh bGxsb2Nrcw0KUHJvY2VzcyAyMjA0IChkYXRhcGxheSkgdGhyZWFkIDB4ZmZm ZmZmMDBiMTcyNmEwMDAgKDEwMDIxNCkNCmV4Y2x1c2l2ZSBzbGVlcCBtdXRl eCBpbnAgKHVkcGlucCkgZiA9IDAgKDB4ZmZmZmZmMDBjYzkwZmNjOCkgbG9j a2VkIEAgL3Vzci9zcmMvc3lzL25ldGluZXQvdWRwX3VzcnJlcS5jOjc2Mg0K UHJvY2VzcyA2MiAocGFnZWRhZW1vbikgdGhyZWFkIDB4ZmZmZmZmMDBlMzU4 YzI4MCAoMTAwMDQ5KQ0KZXhjbHVzaXZlIHNsZWVwIG11dGV4IFVNQSBsb2Nr IHIgPSAwICgweGZmZmZmZmZmODA2OGJmODApIGxvY2tlZCBAIC91c3Ivc3Jj L3N5cy92bS91bWFfY29yZS5jOjE0OTENCmV4Y2x1c2l2ZSBzbGVlcCBtdXRl eCBHaWFudCByID0gMCAoMHhmZmZmZmZmZjgwNjJlZDgwKSBsb2NrZWQgQCAv dXNyL3NyYy9zeXMvdm0vdm1fcGFnZW91dC5jOjcxNw0KUHJvY2VzcyA0OCAo c3dpMTpuZXQpIHRocmVhZCAweGZmZmZmZjAwZTM1OTc3ODAgKDEwMDAyNykN CmV4Y2x1c2l2ZSBzbGVlcCBtdXRleCBJUEZXIHN0YXRpYyBydWxlcyByID0g MCAoMHhmZmZmZmZmZjgwNjdhZTUwKSBsb2NrZWQgQCAvdXNyL3NyYy9zeXMv bmV0aW5ldC9pcF9mdzIuYzoxNDkNCg0KPnNoIHBjcHUNCmNwdWlkPTANCmN1 cnJ0aHJlYWQgICAgICA9IDB4ZmZmZmZmMDBlMzU4YzI4MDogcGlkIDYzICJw YWdlZGFlbW9uIg0KY3VycnBjYiAgICAgICAgID0gMHhmZmZmZmZmZmIzNGUz ZDEwDQpmcGN1cnJ0aHJlYWQgICAgPSBub25lDQppZGxlIHRocmVhZCAgICAg PSAweGZmZmZmZjAwZTM1YjYwMDA6IHBpZCAxNCAoaWRsZSBjcHUwKQ0Kc3Bp biBsb2NrcyBoZWxkID0NCg0KPnNoIHBjcHUgMQ0KY3B1aWQ9MQ0KY3VycnRo cmVhZCAgICAgID0gMHhmZmZmZmYwMGUzNThiM2M4MDogcGlkIDEzICJpZGxl IGNwdTEiDQpjdXJycGNiICAgICAgICAgPSAweGZmZmZmZmZmZmZiMzRlN2Qx MA0KZnBjdXJydGhyZWFkICAgID0gbm9uZQ0KaWRsZSB0aHJlYWQgICAgID0g MHhmZmZmZmYwMGUzNThiM2M4MDogcGlkIDEzICJpZGxlIGNwdTEiDQpzcGlu IGxvY2tzIGhlbGQgPQ0KDQo+c2ggcGNwdSAyDQpjcHVpZD0yDQpjdXJydGhy ZWFkICAgICAgPSAweGZmZmZmZjAwZTM1ZTQwMDA6IHBpZCAyNzE1ICJib25u aWUiDQpjdXJycGNiICAgICAgICAgPSAweGZmZmZmZmZmZmZiNjM2ZGQxMA0K ZnBjdXJydGhyZWFkICAgID0gbm9uZQ0KaWRsZSB0aHJlYWQgICAgID0gMHhm ZmZmZmYwMGUzNWIzYTAwOiBwaWQgMTIgImlkbGUgY3B1MiINCnNwaW4gbG9j a3MgaGVsZCA9DQoNCj5zaCBwY3B1IDMgDQpjcHVpZD0zDQpjdXJydGhyZWFk ICAgICAgPSAweGZmZmZmZjAwZTM1YWVhMDA6IHBpZCA0MCAiaXJxMjc6IGVt MSBlbTIiDQpjdXJycGNiICAgICAgICAgPSAweGZmZmZmZmZmZmZiMzRiNmQx MA0KZnBjdXJydGhyZWFkICAgID0gbm9uZQ0KaWRsZSB0aHJlYWQgICAgID0g MHhmZmZmZmYwMGUzNWIzNzgwOiBwaWQgMTEgImlkbGUgY3B1MCINCnNwaW4g bG9ja3MgaGVsZCA9DQoNCg0KDQpLR0RCOg0KVW5yZWFkIHBvcnRpb24gb2Yg dGhlIGtlcm5lbCBtZXNzYWdlIGJ1ZmZlcjoNCnBhbmljOiBObyBUSUQgYml0 bWFwPw0KY3B1aWQgPSAwDQpLREI6IGVudGVyOiBwYW5pYw0KDQojMCAgZG9h ZHVtcCAoKSBhdCBwY3B1Lmg6MTY3DQoxNjcgICAgIHBjcHUuaDogTm8gc3Vj aCBmaWxlIG9yIGRpcmVjdG9yeS4NCiAgICAgICAgaW4gcGNwdS5oDQooa2dk YikgYnQNCiMwICBkb2FkdW1wICgpIGF0IHBjcHUuaDoxNjcNCiMxICAweGZm ZmZmZmZmODAxOTI0ZjYgaW4gZGJfZm5jYWxsIChkdW1teTE9MCwgZHVtbXky PTAsIGR1bW15Mz0wLCBkdW1teTQ9MHgwKSBhdCAvdXNyL3NyYy9zeXMvZGRi L2RiX2NvbW1hbmQuYzo1MzENCiMyICAweGZmZmZmZmZmODAxOTI5ODUgaW4g ZGJfY29tbWFuZF9sb29wICgpIGF0IC91c3Ivc3JjL3N5cy9kZGIvZGJfY29t bWFuZC5jOjM0OQ0KIzMgIDB4ZmZmZmZmZmY4MDE5NDgzMyBpbiBkYl90cmFw ICh0eXBlPS0xMjg2NzE5NjQ4LCBjb2RlPTApIGF0IC91c3Ivc3JjL3N5cy9k ZGIvZGJfbWFpbi5jOjIyMQ0KIzQgIDB4ZmZmZmZmZmY4MDJjYjhmMCBpbiBr ZGJfdHJhcCAodHlwZT0zLCBjb2RlPTAsIHRmPTB4MCkgYXQgL3Vzci9zcmMv c3lzL2tlcm4vc3Vicl9rZGIuYzo0NzANCiM1ICAweGZmZmZmZmZmODA0MTY5 ZGMgaW4gdHJhcCAoZnJhbWU9DQogICAgICB7dGZfcmRpID0gMCwgdGZfcnNp ID0gLTIxMzY5MjgyNTYsIHRmX3JkeCA9IDAsIHRmX3JjeCA9IDUyMzc3Niwg dGZfcjggPSAtMTI4NjcxOTQ0MCwgdGZfcjkgPSAxMCwgdGZfcmF4ID0gMTgs IHRmX3JieCA9IC0yMTQyNjg2MjU4LCB0Zl9yYnAgPSAtMTI4NjcxOTIwMCwg dGZfcjEwID0gMjA3NjUsIHRmX3IxMSA9IDAsIHRmX3IxMiA9IDAsIHRmX3Ix MyA9IDI1NiwgdGZfcjE0ID0gLTEwOTU2OTczODI3ODQsIHRmX3IxNSA9IDc2 ODYwNSwgdGZfdHJhcG5vID0gMywgdGZfYWRkciA9IDAsIHRmX2ZsYWdzID0g MjU2LCB0Zl9lcnIgPSAwLCB0Zl9yaXAgPSAtMjE0NDU1NDE2MSwgdGZfY3Mg PSA4LCB0Zl9yZmxhZ3MgPSA2NDIsIHRmX3JzcCA9IC0xMjg2NzE5MjAwLCB0 Zl9zcyA9IDE2fSkgYXQgL3Vzci9zcmMvc3lzL2FtZDY0L2FtZDY0L3RyYXAu Yzo0MzENCiM2ICAweGZmZmZmZmZmODA0MDQ2ZmIgaW4gY2FsbHRyYXAgKCkg YXQgL3Vzci9zcmMvc3lzL2FtZDY0L2FtZDY0L2V4Y2VwdGlvbi5TOjE3MQ0K IzcgIDB4MDAwMDAwMDAwMDAwMDAwMCBpbiA/PyAoKQ0KIzggIDB4ZmZmZmZm ZmY4MGExMTAwMCBpbiA/PyAoKQ0KIzkgIDB4MDAwMDAwMDAwMDAwMDAwMCBp biA/PyAoKQ0KIzEwIDB4MDAwMDAwMDAwMDA3ZmUwMCBpbiA/PyAoKQ0KIzEx IDB4ZmZmZmZmZmZiMzRlMzgzMCBpbiA/PyAoKQ0KIzEyIDB4MDAwMDAwMDAw MDAwMDAwYSBpbiA/PyAoKQ0KIzEzIDB4MDAwMDAwMDAwMDAwMDAxMiBpbiA/ PyAoKQ0KIzE0IDB4ZmZmZmZmZmY4MDQ5MzNjZSBpbiBfX2Z1bmNfXy4wICgp DQojMTUgMHhmZmZmZmZmZmIzNGUzOTIwIGluID8/ICgpDQojMTYgMHgwMDAw MDAwMDAwMDA1MTFkIGluID8/ICgpDQojMTcgMHgwMDAwMDAwMDAwMDAwMDAw IGluID8/ICgpDQojMTggMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQoj MTkgMHgwMDAwMDAwMDAwMDAwMTAwIGluID8/ICgpDQojMjAgMHhmZmZmZmYw MGUzNThjMjgwIGluID8/ICgpDQojMjEgMHgwMDAwMDAwMDAwMGJiYTVkIGlu ID8/ICgpDQojMjIgMHgwMDAwMDAwMDAwMDAwMDAzIGluID8/ICgpDQojMjMg MHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojMjQgMHgwMDAwMDAwMDAw MDAwMTAwIGluID8/ICgpDQojMjUgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ ICgpDQojMjYgMHhmZmZmZmZmZjgwMmNiMzRmIGluIGtkYl9lbnRlciAobXNn PTB4MCkgYXQgY3B1ZnVuYy5oOjU5DQojMjcgMHhmZmZmZmZmZjgwMmIwMTg5 IGluIHBhbmljIChmbXQ9MHhmZmZmZmZmZjgwNDkzM2NlICJObyBUSUQgYml0 bWFwPyIpIGF0IC91c3Ivc3JjL3N5cy9rZXJuL2tlcm5fc2h1dGRvd24uYzo1 NTINCiMyOCAweGZmZmZmZmZmODAyYmI2ZDkgaW4gdGhyZWFkX2ZpbmkgKG1l bT0weDAsIHNpemU9MCkgYXQgL3Vzci9zcmMvc3lzL2tlcm4va2Vybl90aHJl YWQuYzoyNjkNCiMyOSAweGZmZmZmZmZmODAzZjk1ZTkgaW4gem9uZV9kcmFp biAoem9uZT0weDEpIGF0IC91c3Ivc3JjL3N5cy92bS91bWFfY29yZS5jOjc0 OQ0KIzMwIDB4ZmZmZmZmZmY4MDNmNzRkNiBpbiB6b25lX2ZvcmVhY2ggKHpm dW5jPTB4ZmZmZmZmZmY4MDNmOTQ0MCA8em9uZV9kcmFpbj4pIGF0IC91c3Iv c3JjL3N5cy92bS91bWFfY29yZS5jOjE0OTQNCiMzMSAweGZmZmZmZmZmODAz ZmE5MTEgaW4gdW1hX3JlY2xhaW0gKCkgYXQgL3Vzci9zcmMvc3lzL3ZtL3Vt YV9jb3JlLmM6MjYyMw0KIzMyIDB4ZmZmZmZmZmY4MDNmNTBlYSBpbiB2bV9w YWdlb3V0ICgpIGF0IC91c3Ivc3JjL3N5cy92bS92bV9wYWdlb3V0LmM6NzI1 DQojMzMgMHhmZmZmZmZmZjgwMjk5ZmEzIGluIGZvcmtfZXhpdCAoY2FsbG91 dD0weGZmZmZmZmZmODAzZjRkYjAgPHZtX3BhZ2VvdXQ+LCBhcmc9MHgwLCBm cmFtZT0weGZmZmZmZmZmYjM0ZTNjNTApDQogICAgYXQgL3Vzci9zcmMvc3lz L2tlcm4va2Vybl9mb3JrLmM6NzkxDQojMzQgMHhmZmZmZmZmZjgwNDA0OGZl IGluIGZvcmtfdHJhbXBvbGluZSAoKSBhdCAvdXNyL3NyYy9zeXMvYW1kNjQv YW1kNjQvZXhjZXB0aW9uLlM6Mjk2DQojMzUgMHgwMDAwMDAwMDAwMDAwMDAw IGluID8/ICgpDQojMzYgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQoj MzcgMHgwMDAwMDAwMDAwMDAwMDAxIGluID8/ICgpDQojMzggMHgwMDAwMDAw MDAwMDAwMDAwIGluID8/ICgpDQojMzkgMHgwMDAwMDAwMDAwMDAwMDAwIGlu ID8/ICgpDQojNDAgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNDEg MHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNDIgMHgwMDAwMDAwMDAw MDAwMDAwIGluID8/ICgpDQojNDMgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ ICgpDQojNDQgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNDUgMHgw MDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNDYgMHgwMDAwMDAwMDAwMDAw MDAwIGluID8/ICgpDQojNDcgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgp DQojNDggMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNDkgMHgwMDAw MDAwMDAwMDAwMDAwIGluID8/ICgpDQojNTAgMHgwMDAwMDAwMDAwMDAwMDAw IGluID8/ICgpDQojNTEgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQoj NTIgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNTMgMHgwMDAwMDAw MDAwMDAwMDAwIGluID8/ICgpDQojNTQgMHgwMDAwMDAwMDAwMDAwMDAwIGlu ID8/ICgpDQojNTUgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNTYg MHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNTcgMHgwMDAwMDAwMDAw MDAwMDAwIGluID8/ICgpDQojNTggMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ ICgpDQojNTkgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNjAgMHgw MDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNjEgMHgwMDAwMDAwMDAwMDAw MDAwIGluID8/ICgpDQojNjIgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgp DQojNjMgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQojNjQgMHgwMDAw MDAwMDAwMDAwMDAwIGluID8/ICgpDQojNjUgMHgwMDAwMDAwMDAwMDAwMDAw IGluID8/ICgpDQojNjYgMHgwMDAwMDAwMDAwMDAwMDAwIGluID8/ICgpDQoj NjcgMHgwMDAwMDAwMDAwODY4MDAwIGluID8/ICgpDQojNjggMHgwMDAwMDAw MDAwMGJiYTVkIGluID8/ICgpDQojNjkgMHgwMDAwMDAwMDAwMDAwMDAxIGlu ID8/ICgpDQojNzAgMHhmZmZmZmYwMGUzNTcxNWQwIGluID8/ICgpDQojNzEg MHhmZmZmZmYwMDNhODk3YzgwIGluID8/ICgpDQojNzIgMHhmZmZmZmZmZmIz NGUzOWQwIGluID8/ICgpDQojNzMgMHhmZmZmZmZmZmIzNGUzOWE4IGluID8/ ICgpDQojNzQgMHhmZmZmZmYwMGUzNThjMjgwIGluID8/ICgpDQojNzUgMHhm ZmZmZmZmZjgwMmMzNDJlIGluIHNjaGVkX3N3aXRjaCAodGQ9MHgwLCBuZXd0 ZD0weGZmZmZmZmZmODAzZjRkYjAsIGZsYWdzPTEpIGF0IC91c3Ivc3JjL3N5 cy9rZXJuL3NjaGVkXzRic2QuYzo4ODENClByZXZpb3VzIGZyYW1lIGlubmVy IHRvIHRoaXMgZnJhbWUgKGNvcnJ1cHQgc3RhY2s/KQ0KKGtnZGIpIGZyYW1l IDI4DQojMjggMHhmZmZmZmZmZjgwMmJiNmQ5IGluIHRocmVhZF9maW5pICht ZW09MHgwLCBzaXplPTApIGF0IC91c3Ivc3JjL3N5cy9rZXJuL2tlcm5fdGhy ZWFkLmM6MjY5DQoyNjkgICAgICAgICAgICAgS0FTU0VSVChibXAgIT0gTlVM TCwgKCJObyBUSUQgYml0bWFwPyIpKTsNCihrZ2RiKSBsDQoyNjQgICAgICAg ICAgICAgU1RBSUxRX0ZPUkVBQ0goYm1wLCAmdGlkX2JpdG1hcCwgYm1wX25l eHQpIHsNCjI2NSAgICAgICAgICAgICAgICAgICAgIGlmICh0ZC0+dGRfdGlk ID49IGJtcC0+Ym1wX2Jhc2UgJiYNCjI2NiAgICAgICAgICAgICAgICAgICAg ICAgICB0ZC0+dGRfdGlkIDwgYm1wLT5ibXBfYmFzZSArIFRJRF9JRFNfUEVS X1BBUlQpDQoyNjcgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJyZWFr Ow0KMjY4ICAgICAgICAgICAgIH0NCjI2OSAgICAgICAgICAgICBLQVNTRVJU KGJtcCAhPSBOVUxMLCAoIk5vIFRJRCBiaXRtYXA/IikpOw0KMjcwICAgICAg ICAgICAgIG10eF9sb2NrKCZ0aWRfbG9jayk7DQoyNzEgICAgICAgICAgICAg dGlkID0gdGQtPnRkX3RpZCAtIGJtcC0+Ym1wX2Jhc2U7DQoyNzIgICAgICAg ICAgICAgaWR4ID0gdGlkIC8gVElEX0lEU19QRVJfSURYOw0KMjczICAgICAg ICAgICAgIGJpdCA9IDFVTCA8PCAodGlkICUgVElEX0lEU19QRVJfSURYKTsN CihrZ2RiKSBwIGJtcA0KJDEgPSAoc3RydWN0IHRpZF9iaXRtYXBfcGFydCAq KSAweDANCihrZ2RiKSBpIHJlZw0KcmF4ICAgICAgICAgICAgMHgwICAgICAg MA0KcmJ4ICAgICAgICAgICAgMHgwICAgICAgMA0KcmN4ICAgICAgICAgICAg MHgwICAgICAgMA0KcmR4ICAgICAgICAgICAgMHgwICAgICAgMA0KcnNpICAg ICAgICAgICAgMHgwICAgICAgMA0KcmRpICAgICAgICAgICAgMHgwICAgICAg MA0KcmJwICAgICAgICAgICAgMHgwICAgICAgMHgwDQpyc3AgICAgICAgICAg ICAweGZmZmZmZmZmYjM0ZTNjNTAgICAgICAgMHhmZmZmZmZmZmIzNGUzYzUw DQpyOCAgICAgICAgICAgICAweDAgICAgICAwDQpyOSAgICAgICAgICAgICAw eDAgICAgICAwDQpyMTAgICAgICAgICAgICAweDAgICAgICAwDQpyMTEgICAg ICAgICAgICAweDAgICAgICAwDQpyMTIgICAgICAgICAgICAweGZmZmZmZmZm ODAzZjRkYjAgICAgICAgLTIxNDMzMzQ5OTINCnIxMyAgICAgICAgICAgIDB4 ZmZmZmZmZmY4MDYyOGVhMCAgICAgICAtMjE0MTAyNDYwOA0KcjE0ICAgICAg ICAgICAgMHgxICAgICAgMQ0KcjE1ICAgICAgICAgICAgMHgwICAgICAgMA0K cmlwICAgICAgICAgICAgMHhmZmZmZmZmZjgwNDA0OGZlICAgICAgIDB4ZmZm ZmZmZmY4MDQwNDhmZSA8Zm9ya190cmFtcG9saW5lKzE0Pg0KZWZsYWdzICAg ICAgICAgMHg4MiAgICAgMTMwDQpjcyAgICAgICAgICAgICAweDAgICAgICAw DQpzcyAgICAgICAgICAgICAweDAgICAgICAwDQpkcyAgICAgICAgICAgICAw eDAgICAgICAwDQplcyAgICAgICAgICAgICAweDAgICAgICAwDQpmcyAgICAg ICAgICAgICAweDAgICAgICAwDQpncyAgICAgICAgICAgICAweDAgICAgICAw DQooa2dkYikgbCAqMHhmZmZmZmZmZjgwNDA0OGZlDQoweGZmZmZmZmZmODA0 MDQ4ZmUgaXMgYXQgL3Vzci9zcmMvc3lzL2FtZDY0L2FtZDY0L2V4Y2VwdGlv bi5TOjI5OC4NCjI5MyAgICAgICAgICAgICBtb3ZxICAgICVyMTIsICVyZGkg ICAgICAgICAgICAgIC8qIGZ1bmN0aW9uICovDQoyOTQgICAgICAgICAgICAg bW92cSAgICAlcmJ4LCAlcnNpICAgICAgICAgICAgICAvKiBhcmcxICovDQoy OTUgICAgICAgICAgICAgbW92cSAgICAlcnNwLCAlcmR4ICAgICAgICAgICAg ICAvKiB0cmFwZnJhbWUgcG9pbnRlciAqLw0KMjk2ICAgICAgICAgICAgIGNh bGwgICAgZm9ya19leGl0DQoyOTcgICAgICAgICAgICAgTUVYSVRDT1VOVA0K Mjk4ICAgICAgICAgICAgIGptcCAgICAgZG9yZXRpICAgICAgICAgICAgICAg ICAgLyogSGFuZGxlIGFueSBBU1RzICovDQoyOTkNCjMwMCAgICAgLyoNCjMw MSAgICAgICogVG8gZWZmaWNpZW50bHkgaW1wbGVtZW50IGNsYXNzaWZpY2F0 aW9uIG9mIHRyYXAgYW5kIGludGVycnVwdCBoYW5kbGVycw0KMzAyICAgICAg KiBmb3IgcHJvZmlsaW5nLCB0aGVyZSBtdXN0IGJlIG9ubHkgdHJhcCBoYW5k bGVycyBiZXR3ZWVuIHRoZSBsYWJlbHMgYnRyYXANCg0K --0-432868459-1128013586=:65402--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050929124027.U65402>