Date: Fri, 12 Dec 2014 19:34:47 +0000 From: Roger Leigh <rleigh@codelibre.net> To: stable@FreeBSD.org Subject: Hard system lockups with 10.1, probably drm/newcons/radeonkms-related Message-ID: <20141212193447.GA1657@codelibre.net>
next in thread | raw e-mail | index | archive | help
--3uo+9/B/ebqu+fSQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi folks, With 10.1-RELEASE, I've enabled newcons at boot with kern.vty="vt" in loader.conf. With the latest Xorg/drm installed with pkg, I'm seeing intermittent hangs and hard lockups of the system. I've included the logs for one which recovered earlier today, but later on it just locked up completely and I don't have logs for that since I had to do a hard reset. I had to install and enable hal+dbus to get a working keyboard and mouse when running X, despite both working fine on the console! Not sure what the trigger is. Possibly also related to input. The first hard hang was after logging in with "mwm" via kdm4. It didn't start mwm, so I ran "mwm&" in the xterm; it locked up when I clicked and dragged the window title, i.e. when initiating the drag event. The second hang was while typing into a tmux session inside a konsole window. Nothing particularly special happening at the moment it locked up. I'm happy to do further debugging, but given that it locks up the whole system, I'm not sure how to go about getting any useful information at that point. The graphics card is an AMD Radeon HD 6800 Series using /dev/dri/card0. Starting X11 automatically loads the needed modules: # kldstat Id Refs Address Size Name 1 59 0xffffffff80200000 1755658 kernel 2 1 0xffffffff81956000 267f48 zfs.ko 3 2 0xffffffff81bbe000 6780 opensolaris.ko 4 1 0xffffffff81c11000 2b58 uhid.ko 5 1 0xffffffff81c14000 357f ums.ko 6 2 0xffffffff81c18000 28c0 vboxnetflt.ko 7 2 0xffffffff81c1b000 b998 netgraph.ko 8 2 0xffffffff81c27000 434c0 vboxdrv.ko 9 1 0xffffffff81c6b000 40a7 ng_ether.ko 10 1 0xffffffff81c70000 3ec0 vboxnetadp.ko 11 1 0xffffffff81c74000 11a57a radeonkms.ko 12 1 0xffffffff81d8f000 47f80 drm2.ko 13 4 0xffffffff81dd7000 1ff2 iicbus.ko 14 1 0xffffffff81dd9000 1a46 iic.ko 15 1 0xffffffff81ddb000 1e48 iicbb.ko 16 1 0xffffffff81ddd000 18f3 radeonkmsfw_BARTS_pfp.ko 17 1 0xffffffff81ddf000 1ce8 radeonkmsfw_BARTS_me.ko 18 1 0xffffffff81de1000 136f radeonkmsfw_BTC_rlc.ko 19 1 0xffffffff81de3000 6585 radeonkmsfw_BARTS_mc.ko Kernel log for the recoverable hang: Dec 12 13:23:23 sorilea kernel: drmn0: error: GPU lockup CP stall for more than 10000m sec Dec 12 13:23:23 sorilea kernel: drmn0: warning: GPU lockup (waiting for 0x000000000008 7184 last fence id 0x0000000000087177) Dec 12 13:23:23 sorilea kernel: drmn0: info: Saved 407 dwords of commands on ring 0. Dec 12 13:23:23 sorilea kernel: drmn0: info: GPU softreset: 0x00000003 Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_STATUS = 0xA0003828 Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_STATUS_SE0 = 0x00000007 Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_STATUS_SE1 = 0x00000007 Dec 12 13:23:23 sorilea kernel: drmn0: info: SRBM_STATUS = 0x200000C0 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_008674_CP_STALLED_STAT1 = 0x00000000 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_008678_CP_STALLED_STAT2 = 0x00010100 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_00867C_CP_BUSY_STAT = 0x00020182 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_008680_CP_STAT = 0x80038243 Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_SOFT_RESET=0x00007F6B Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_STATUS = 0x00003828 Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_STATUS_SE0 = 0x00000007 Dec 12 13:23:23 sorilea kernel: drmn0: info: GRBM_STATUS_SE1 = 0x00000007 Dec 12 13:23:23 sorilea kernel: drmn0: info: SRBM_STATUS = 0x200000C0 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_008674_CP_STALLED_STAT1 = 0x00000000 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_008678_CP_STALLED_STAT2 = 0x00000000 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_00867C_CP_BUSY_STAT = 0x00000000 Dec 12 13:23:23 sorilea kernel: drmn0: info: R_008680_CP_STAT = 0x00000000 Dec 12 13:23:23 sorilea kernel: drmn0: info: GPU reset succeeded, trying to resume Dec 12 13:23:23 sorilea kernel: info: [drm] probing gen 2 caps for device 1002:5a16 = 2/0 Dec 12 13:23:23 sorilea kernel: info: [drm] enabling PCIE gen 2 link speeds, disable w ith radeon.pcie_gen2=0 Dec 12 13:23:23 sorilea kernel: info: [drm] PCIE GART of 512M enabled (table at 0x0000 000000040000). Dec 12 13:23:23 sorilea kernel: drmn0: info: WB enabled Dec 12 13:23:23 sorilea kernel: drmn0: info: fence driver on ring 0 use gpu addr 0x000 0000040000c00 and cpu addr 0x0xfffff8007e940c00 Dec 12 13:23:23 sorilea kernel: drmn0: info: fence driver on ring 3 use gpu addr 0x000 0000040000c0c and cpu addr 0x0xfffff8007e940c0c Dec 12 13:23:23 sorilea kernel: info: [drm] ring test on 0 succeeded in 4 usecs Dec 12 13:23:23 sorilea kernel: info: [drm] ring test on 3 succeeded in 2 usecs Dec 12 13:23:33 sorilea kernel: drmn0: error: GPU lockup CP stall for more than 10000m sec Dec 12 13:23:33 sorilea kernel: drmn0: warning: GPU lockup (waiting for 0x000000000008 7185 last fence id 0x0000000000087177) Dec 12 13:23:33 sorilea kernel: error: [drm:pid939:r600_ib_test] *ERROR* radeon: fence wait failed (-11). Dec 12 13:23:33 sorilea kernel: error: [drm:pid939:radeon_ib_ring_tests] *ERROR* radeon: failed testing IB on GFX ring (-11). Dec 12 13:23:33 sorilea kernel: drmn0: error: ib ring test failed (-11). Dec 12 13:23:33 sorilea kernel: drmn0: info: GPU softreset: 0x00000003 Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_STATUS = 0xA0003828 Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_STATUS_SE0 = 0x00000007 Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_STATUS_SE1 = 0x00000007 Dec 12 13:23:33 sorilea kernel: drmn0: info: SRBM_STATUS = 0x200000C0 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_008674_CP_STALLED_STAT1 = 0x00000000 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_008678_CP_STALLED_STAT2 = 0x00004100 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_00867C_CP_BUSY_STAT = 0x00020182 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_008680_CP_STAT = 0x80028243 Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_SOFT_RESET=0x00007F6B Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_STATUS = 0x00003828 Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_STATUS_SE0 = 0x00000007 Dec 12 13:23:33 sorilea kernel: drmn0: info: GRBM_STATUS_SE1 = 0x00000007 Dec 12 13:23:33 sorilea kernel: drmn0: info: SRBM_STATUS = 0x200000C0 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_008674_CP_STALLED_STAT1 = 0x00000000 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_008678_CP_STALLED_STAT2 = 0x00000000 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_00867C_CP_BUSY_STAT = 0x00000000 Dec 12 13:23:33 sorilea kernel: drmn0: info: R_008680_CP_STAT = 0x00000000 Dec 12 13:23:33 sorilea kernel: drmn0: info: GPU reset succeeded, trying to resume Dec 12 13:23:33 sorilea kernel: info: [drm] probing gen 2 caps for device 1002:5a16 = 2/0 Dec 12 13:23:33 sorilea kernel: info: [drm] enabling PCIE gen 2 link speeds, disable with radeon.pcie_gen2=0 Dec 12 13:23:33 sorilea kernel: info: [drm] PCIE GART of 512M enabled (table at 0x0000000000040000). Dec 12 13:23:33 sorilea kernel: drmn0: info: WB enabled Dec 12 13:23:33 sorilea kernel: drmn0: info: fence driver on ring 0 use gpu addr 0x0000000040000c00 and cpu addr 0x0xfffff8007e940c00 Dec 12 13:23:33 sorilea kernel: drmn0: info: fence driver on ring 3 use gpu addr 0x0000000040000c0c and cpu addr 0x0xfffff8007e940c0c Dec 12 13:23:33 sorilea kernel: info: [drm] ring test on 0 succeeded in 4 usecs Dec 12 13:23:33 sorilea kernel: info: [drm] ring test on 3 succeeded in 2 usecs Dec 12 13:23:33 sorilea kernel: info: [drm] ib test on ring 0 succeeded in 0 usecs Dec 12 13:23:33 sorilea kernel: info: [drm] ib test on ring 3 succeeded in 1 usecs It worked perfectly for 5 hours after this recovery. Thanks all, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `- GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800 --3uo+9/B/ebqu+fSQ Content-Type: application/octet-stream Content-Disposition: attachment; filename="Xorg.0.log.old.xz" Content-Transfer-Encoding: base64 /Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4PcBIXtdAC2IMEZDZYdxGZ3Q7R97rK/hhwP44Yww 1U2aTYg/bTlXRj+VzmJ47H6G5BI3bBFuBnAWqxfjrS3XAdDml0uHt6Q7x5YeJt3MzH7GTqYy lKN4tULXowkp+fuwnVJgaIz8a794I0mqjxti6r+uf/DaL5rqHafgXCn/8R1wm7x+Dwfx56Uw wvgZqab+BqAMIpByplr8DONcAoZnJ6fhf/WcB3P5/yaFBvNE6CtH3Ll2rJz/YoYEpWgvGtz2 rD8Syvw0ZvW+klIeIOnFpB73RboSM8N6syvYTNR/Jdv6bDvICCo1QR1VbL5t9LE7XYvs0GJE CfQCUQ3V4JmVCRaw6ZIgxnmb3XSjGiPdJNI/Zb1LKw1daeHVz63QIOJJZlBt67GRGqDX8a7B B4NXu0aMbWdHf75lEKRz/5dtW0XhM/l2jNE4lrjGPTIl6aSF4dOV7AcxbM9X1avKuo3G5EXT epCZ6znEFF/aBgaCCk2+p1A20/QIbC7vOP8lL/hqqvTiiE78ldl4b4qlABf0cVH4UZZFhxNS O+ZpoUDUbur7DHCLNDyrS/Ma90W3bpiLhPNJFJNDx/iMl0b5uqml6pol1hvPZ904qOJVKwbD zc8nSB0UZR4T1QeMEaLUiTKy1p1QvbIloFZDigHNFIGab/MFdsgqGuaqh5aqXenW9B8AUIZn HkmhKt0SbqHUqKJlZcmzi+5h2URtpXvXVA9I8bGKFD8IADR8/UYbpUEEcGRSESoWqcEAnwEi LSdqza0ZMkYDjUtyw/25ZwMU56t7s8LBbB5iLhqjHTRr7wA5PbZYI+wSbF/iMrGYRtzBz/RW +/1yZfjrjTj7de0CqWMjpmApqZBJg4hVxwErSOyhiJM5K5gX1qg4stTUc1LB/vD88hFvxSTB xozakgjrIdUVJYZ7s7SeSfEDjAzzsOwHKiOtdQ4Zc3tf8E0xoGGjWHwf4dBvCQkpHb+3wpHq 0yw+UY0h/maffw+iIsk5qUw0DQ5EgNboqbRRfd0DvZSBanOu//5OighpptcLP11oqeON4g7X TCSWRXOlm/px7j5PAjwjglD86GMYu0eVZaCzhexWK8iAGf3ebYOPPiGkolRaqiG7OjXor2w6 y1QEs9gbRd7odfEIMoz1c6vGJ+fojSgq/W7/nJkpjY8PrIKc5gyZrLAOkE3fqq1zOJpQ4gbh YkmGpgA/dNYRObDIS7x29whMyaGINtVEgM9DkNpPVSJMAl1ou9iaoZ0XH6A58JmUycXdDn18 Eyyq8aOcsdw22giam1c4O/UgyhUUW+LpODXER4eSVjYV7DACzKwT98o7LyP5M03JhhqOT03+ AXmQJ01ykMQT4ZRLi7v0SGX/PqxscwEejL/3mIgiRdGGMddxenzk16r/m7v85M0D4xwovCl8 dMZI4qS/qEiwQ42F5Wv0SJ2hV9yZZWGXl3+OxNdxw+j6ykUDGLmkJZRc6uie6n/nouTWb1q+ 0gF8oWDUNgXGfUk+/nmk1lkmn3nFgBsd0vU/vrRrU4nnlyNUsCCRtw962Y2t6qg2jRS1ZYqn DMce/lF3jrOUpddHRJek8tOPW9vJRyG+GVzyx+kTktTNK3tU1wTPEYhV9uXLcyv7nL4aMvE+ Xvtpy3T8gZohvSGGGhLS9VJv6bCr0NlGm/XtZeAh32Xxn9D2mcpguJbdCHgzfmPfQQQPs8Cx PQMzkc7sauvDpP2mrRdZ8V3PpgjmRF6cD30hNF6dOrqmjHQn+mfS6cpPhJOnwvUodi/liz2e pWhOmDiz1XZALX1lJ7CCVZnbmneTL65zqEEaQ8PoYyd4pCNk1Cr11BNy3XGFTRuGnIqytcu9 bDTYgTmDXoAlxN2r49Vz3OXnNpbqZf3TD2rhIgiyvy3qNl2cCphazeGb1yhvtAVsoy2TTS6e Ndj6WU4YoxlUOdghcBfyt+xMeRNKAcIDw0wI06aqk3lP5fq/2sls2crTp6eOhkdnicNl+lir 6dYLYf2wpvRBy+faDNb+wHYzNpwD6UAHf/m0BPLuNUpz5WkmkEDGjuDyvT972FaAR9FM9Uy1 3tTZygO8FZZSXBRpVQU0gUvxrFcUGzOlXu2lcEi1bZ37Io0fDuLORSFOFdSb4M76OxVLzzZ6 dEHOQrzp47BpZPmgGq2Gd7ImsklotiXYEG8/qNdHO8FGDlNQXEDknz7+IvfFxndc7cs701Af wwY9kQjOOP8jr8WRvQKayyJAVgh1g5Mit8f/oblMXvTxmyGbVI6fI0LMc3TIxUpiNJVxlZmd hhZLVvQ2fqOBV4xv2MBaYOLdykDzABEfKy0lnj84+sqo+Y2ccgQxmF/3MSyrXfrk/x9KihDC 5tGhg39jqIR1TZfEOnZtbToKV2wT/lFKNiA7MaOoCfBGUmmMt9MymvCcqALalAxko235wbj4 07o4sMA8rjfRigCitCyjiFi575LfI/HBFuwx5mLtU11JBwFSEptmC+rfUylKPel6ztC70n4c Miz4vug/L7n57QbP3xDJqzX/HhTJ1ZXsKKDVQ8PW45TtEOvEABlIUPVhbKx3bAavBNcaV8Lm 91++PmifLilTUbO9oq2nLEHYEPuH4X27xqfwDyJ10CItoPdHXsb3arOP/p4wZgDGLx01y12K U9a9O6z7fxRWKxJiE7ih6OfQjq0hwY0/DFaS8x6MUsBzAUIinVI+72isnwvIgnFiVfLeew+v RPwNAcbQ6EV7ZeyegnE+gUd1fCSyKq8GqsK24w5QurYg8Gvfy7zsaTooPeSuRoNJXkXw/HVn 4GTd6AXqx8GC6wXPnAgrk/djjk3XmvWVM211ZjUrMRmjcMUj+vjg5d789b/7c2Q7c5y51XJi CWs7OmQd/JOYP000Lg8ICLyehmVY4mLeMlnufhYZiDJvbq5CvJVXv9ubUneaQTgTpEJlIBe1 tl8DcNqJHDOlP52H/u29geaWrXjlnakpuacrt+A/X7taAeDHon6QD3flnt6HOxkOiZeQqJHe Vx+73nZRDuEAQE3pX2/3NcJfPKqBoJMnbf5QiZaDKAWMKK6sStlMLdOkZ1BakuC0R6F4CIdy j5MF+6/cGji0mfKhgDlatpMPjETJNCiEs+tf9SRIsJKvTQw+3tsPgXjibL2Q0I1o3gSE/NTL uMeaG4k/W3HlNMM7YKXoZq0m97oESktRtLZguB6FB4v0ORJ3wBlsISQD0z6bVZmHzIZE33Fr FFEyp5G4z1TzUzsd1oQDlNDrxdnDpQWbUeqtHFlzZzHuEPQqWvfL+SnZSnQ4pnTkBkFIlrtu v3laQKJS6FH6+Lu4ZvFiFuai+YZGJX08EFruuwR1ImqWnuitbsYWLJu2nCNfuta4DEF1mw+W 224vbUuMYNR04YG/NInNgQwIK0B/5ZSw4y65WsPGMicpyiLln0vbJR/GnM0AqlVHGgT2w250 pe57iOrtYSfg5se0t0K/GTyOBvXixteZ9Z/DizVwWldNEXAlHVpjY48B2If2isMM+WXiEP2O 8EI4u61lw5Cc/X/8Y4YxQepGWKScV8WN2Pk7rm6/B4ukBwpa2o2krUnOQgVEI1YJnQCZgBB1 wKc8afg1zBYHuBwpW8xVONabo9PSzFU+dNia2ButKrTru7Wib0nsbYbrmkOWe9I3f4GZktxK 0eVIwT1bKaTJu9pIXIcigiQD+bSNH3uhZg1QHvJw4gBrBzl8fKx3GwkT3XpUiXwem9woQkXa L1XAytsL4dAuCsvhBcZb0LiZoa3jRB/u3drbekXBwFKIlGZsHOdGqU5LeicH3ilSUZrZ/3T4 TjA/YG4ThQjIKhaiU3Sdw1mcI5HB+vmO6I8/lreIfrgpEt00x7w15BZDyNcB2xzUqEZhEOmk aYQ0/9AtozVhdjlCB5QH0mJki81s5ZHHltgLIgG9dUTY/tetzoz4Y9vCOx3aDeqtH5MkTpjr LEXQS/uxKDmNkBJr8rg9gt8HZ+Y2ApEPAg6OpeI0az1enEhhts2gcRirGeBEuPO5LgQmmlgd y+C9qlOQLBtP9If14OkPRZAK8UHvXkBJfl+U6qK0wuVrmIwie2YwG4KURyF8KFEqj0XtTHzA 9Yx0DB8c4MhJMeuNi7eYP/ADaBetrO5zu/VagLPLa4xDZcyHh+zB77YeXU3U/1juakiRERsF 3i3LF1/oc1WmE/2POqNXB31Psb1reQxLBvJuRnyB2G4IMFRLh2UKcofCAq1dKETTcZP7emjJ 7NkZavg0DavueiHZVvimlH1h3IgWFBp+CG8g1uQpx6qW75ih5SqrXdD5UqQRBAVmdie8IVJv rsUiK9pWNLjleDUjsckuu7YIvDS+CLnlr/IYS0K3oczvZPp552/oJaY4nrxbTafg1/25PzPY 7CC/jqvT5bVVZgKo4Tpo+dgYz4aQWM+kn2qewUgruDTMeJebQLIFLUB6mZU2qundqE0GXg+j m9s08rIIYsMFSeLFjEboAYBXs5Akqs9xUzR+fX/kmXpnwRvd4vxOC0xuzzcdgid65ncqwngu nn3444CTfS7qLhS7si4oGvJ6iHapYuq94B5G+yES52Jd8lOFmk/KVPSB1lWFtRvs2ryUe2bc w/AY+JKCr9QUwqEZIxzXLwKGrLEqjzVDrRQQatIYaZuydxJuk4ogIwnnttVD5qOr7cx8GscO cUq/dxcbSaDmuaknW79jxC/kKOUH+ZfZaWAbpfj1fIAgu8Sn7yWYGoFH9Jh9oLqQ/bkuqYyM TpNWuh+qEG2664YmQOpCC+mMeieooGZmcfLWhTzEM8jokK7asvzKOu6nibIWMPeyOAzl+o54 Nd5tv8cx7DiKluFLviG9P0nG6WtaIdn5lwrbFb/uEoRLQciZ69/AFGo+Da1+mBMut8oerUaD rBIOoi66A919NRWw+zkA4AwGbYfkKk61tyN4EsPDGjEH6AwyQBJeFUWKRDwg3gazLnbtJiDF +5ZSDW2EnM5YL+T3eCJBevbQ1nBu9vbjWTsYNW45hRZ4ym1ZqX1RLsUC9sFb9DkpamYue1DP UaJcXcxMbZwDRofyh8eY8QL11GiyOzaVoDbET2yKdOrSqNGsb0FtgL3/JiO8yNYOYJ33qr8J N5jkSVt6J7oOtm4fwK0VcCVq1sf89TsCfCszI4DGo5dy1gaiWPZ4GbMBY4oqSgbVMjUJxE4/ taIyqGQJEYRPqz9gSYyH59aW57+cnCwrXA6aUrR4ccm/DqD1ybT8uxDdif4GfQrZSfEb3Pry 8KSXEJQCBOalc2sl8i/T02YUo1xBxMJiW948mQ/ESNquLKy79VjpzC3Vi1hYeFxQ547P/UPg VhlOmbt3epPAE+BALZdRxA6ylfxFZEvkW4V/48ZVufUIG47N9a91ULYEKhOPK/GBq19hTFSr HU6sUifn9nUTsY2LAh6glFNaXwP59C7YvX3lU8pXtQDeI0sGSOMVfFcnPU1Q8ESWNHb/3WEJ Ydx7SwmzGwKLp0QEbnaqqk0B/d08YQ8fSUrjt85EYGVpEk0oBlSW08kXPlpcoHfGMmC6KGIa xrrKOmqOTge5v+E9hVh1tWKxlRhHdWPISW67MXhWAxTEvcoLZVHmKTG2z9SjCP/ZIwSpDjll iCz9ha1Yt8L7cTJ2FyzlIYBnwolG3s979e0BYsXWCCeU5XGLslYXyd7i1vd3+2C36EUAmYqc MSlBatu20CRyCeQhBUAG183nD5sHba1g/NDv+SsW4a6SI39M0lAExbsCBPuNClO5XeNnQXrU 2sjX2qvOyKWfMc9Jm0pn1RUv9x1UGjp+6lR+mg4Bz0OUufaAShrs8t3cM9jUuk237VlCMz81 8I+mV2uosHrybjfP+Nr9lp/zbr75RCSffKqeOIY0ihiIdL4+30dkxYRzeUuV8fi7NelggHT2 BtepCyq87ZnVMHqRHJF/oPD5WL1yqWgkQWWKdiQHGEnizWd8DTeBqQQgVPi7ekkgS14nSx5/ nN+we8sRuB4+YV3uodnXX/ViS+09hsWZsffiDaBupC0hzsqfUh3bVrO2gl5eBoJRsfW8qqwX tQr0z2aX/DlP4rMwNMVAsiuwz2farUHlTZfSV6mY2oH9BTMw4mDurlThL5WObv1zQvuTXJzR mPp5qF5wQ78sQntfrPmxdeL9aXpVuDXhg3y9xop1yg6fWmsquMgJSu3ZHAimN3k2uqPuA9dW 77NyaQ/OvO81M4RWkNAQUU1p8KaCVDCBJ0YzdFJodwFl1x3heE5a3iisq1MaxZ+nX+pibxqW ScTeI+Hm5GFl/ETPLNZI5nrv1OsJAJdSn6OpWxz8k6TFNuFfcks2+6bTdxK95rYXLU3+efYb QMhJ+rkEtkIb2ZjOWxh37VOgGMTXHCAndeK3tv1Shkt4hpc09bgRD0BwEZa2rxDE0AP804so QDZEr1dPOeE8Oz7LU4X0cOFkA5UA5zhLVpA1DmXCkzgHFjgqcBIRlEYWB8q2Prp1KOYzBWjn LSO55OQVGcArT9znKn5yeSF+9Y9Cu5d3EDX5AjR6kxm1zoEheAPV9V/8AMKvG4M1rUl8/XOR bzK/VYeSERJ2k+tch1Dx/D5mRmctdnz1/ahF3GdF6C6bo6dGlAtZC40JlDlrp8FKITJvFUHK C9TS7V8m8JA+GVr+hfovfdb2Hq4qJWsmAgFjJ7Z30ObMe2coGooL8ttLO8jHf9SQTqj6h5It dxCPt3rWTOKUup3nc2Xgx5EhsQ4g6d0DrXvZGOaqq0mCwcf2HvqnhOxIiAQiKUIvs1g019ve eGyWaBZp7/odZOEidDf3t+/JGHB7AX/yM6An3ga05O5FuZXx/KKsssX7w5BDKI07yUkjfctL xIWE0mtmPaxOnoZp8xtIcdlfQHtnjjyRv6B+7p7PP2b5YOfHUNmkUE3yzLry3eT8tymNl+eq QVq5XiN+WLo0/9pRt3FgWoknS5qzMaynBQO6eqM7YSp29tKadQdKKJ4Apz4Rv1uov2Re0+lN XxkwlSRxIum1f6Lg+j49oGfn3gu1dIieksH7vh1BUh4aBkSF925g0AQ8hJRWZ4cbbPvRzNW4 q+UlrKV062ijzwEMq8EKnWAu8rYjeHlmZFTADgT5li9L2qKt2RznNWOZFrRER8zHfNqQHSSp yLzGeotU6/Psl49YbYfK3aDOv6j/jxOgU3NQ5uUbG1EGxPRcN2fg535V9AwDxqyDCtKFI+Yy ZFaxTzbzlk+YYBjQ4n0Dz9c932GOdwBXlsaE7WvsmP3jJSKEutlD3vqDVZKCwVroFA0FKNT5 2c7DgBDhb59ySY0yHjRlGAhdqjk9+hJl7GO9vqqeji/+RD11DaS1Mf0AFAf3zmmx6wLPc5xA QCARcwpgKS44KWtSCeNvhrcTyUFarf/2Nh2dTYH16WIUxvDy2rip1YKLH//cv5ygiiL7vPTg KAl2vXf+BHbXeNDvQlKwZWekgEBgKJY94IMlkdcfL/jaiReh3psUjPxO6JA/oR3+bL08OX54 5v4Luzq7qnJ3F4H8J024KCyaKXWajxKVE1DIJiQ/B2W1RdrFoS4i5PJ1tIslJstW5b6BBMBF IiA0W/pCuYhPcdy6RV0iJ1grHSvXavKviHhEiMMC4/ZYna9kyhhjpRW0ecT44aWa+erO7XtM NdkNWiW9l13ADQVhh/TXfHbkG7iWG0Z4LvBjEXRU92ehSAY0+kmoZdhsGub8H6DBiHnowTsI MAYYAkefzWdTNnXPvsW/9/9TsdoNUlAvcMKZ0blcHDZzgWBWKQe5iMP+oRUIq3BGbEg385EE bD+aV0txBGM8+z785Akec1qu5sV/m5qZ55fCHzs6xpOZnsyIFElLD78M9RqL3CebA/er0fkk 6G5IMix5gZEt/Nlmy1fgnst9Kg6DkcGManpB6qTBT64mue/hoBcqFsIGuqjgTgi+gmBVZOO7 fZTASzPtFiA8tpafiSIBovfN9WeXN98mkatxCDJWUDi9VTxpHRbrUBRfAJmgWIu8TuIt+0EV vr3pE3k2PmGy/rKdUyK4yoYASCWVZ35I+xreKAiPtnfIV2QMJ5Sea2RzzKeXJVMzL/6DeNAj c9J34pdwxvrkY7zyWLeqBS2m2juU/IKNkwY2tGlDIKZ0kl/ZVECG9Yc8ggL12hZq8W1836xg K3mSx3+RigF41OC9hbpPFii05Qttoo/oHAcdjNeqmEu6gxzlgDs9shu/4DLiqjyloRAvqDPd 3+x0fsOZGhYnmsDHXJi4fIjr7YbgGh+oeKA9ForrlOHSWJYWECgtEi1nHKCWB+T5HBVfJ5LJ hoFKHOQYL7zoJmRNCPn1RRcNTYY/ThLEIJeOveiDlizZfS+iRXdpLASyi2irNXzslmkP8AAI VUgib5TQenwMXB3lPti3jERuXjtLrxa3UV/xhAIRyz1FW6rjeJVTeoYGxKn3vibDAqrV+HSG SeaFUcqvuNv94U2YoBnH2YXHOrKFma6dqBlABPIo+Bpl9dLXloBWx5Mq76iwuN3WBvvpy1pZ XiajMuhmUXSwKFrMBJ/Wzhm5EhXKqD2iTb9KgJm92PV14RQ+o9sq5QMpBHBwo6ok3gymU0wo GfnJKXF8cfwgQGyDNTPzNYF4cG8UZmNsKftqS2HwCtKMgKuAgKyXJdaGDocN6CmXMrewl98l cm+F+P7oIHB3Td5CI6cc+F+RHaQWCY8Ei8ZYUjiGD4IRHk5g4slqXKhFh+p58LG3nKlBtDSM OhCWk0jR4QOCYb6Cx08ZILiOYYu3eWlNCW8etM+ROxt4vRo4+Hm1FParfWpi+szYJaPj28ok pnzSEhs/XlHpwNobHwsH2WvQnRP0oPYuq9FwTAF7mRrHo9//RPT0mTjgQR0drS/PmAYJQA1R 5tfEmqCrfNeGyYQbjLrXJWilFu2Knoo8WPGWTuQQ6PpGjEy4btuxGKqEap9rt+0indnR4Qri S2zRiUqpEwyskYFl4soBuCvQf0w9hUdjCxP8xS6dQNFCMRFSgXrGuFUgMtckE/5BKBbZU+ma XpqW4ug/izrhEgiuJ3X0R2LBXt8FtyZKL5zuXHdvgHhP584YDN8ftDXedfRd10d06wAbq9hc Hm7nr1IPH3jxWhXIfSThqnbX7h/nulb4qkElb6eGYk6Z2ydxKsqWkvXyCU8Iw6he3FdBuHSV CSIefqwwvn8HhgjrnZEKsfBr2aEG1cHHQ+nowubG1AFMmNrxNL1eFXMvPFt+BqCgLXzOsXWf 6mPzGqaufK4oEtg+sfX/EqfqJqrBBGNi5r6Om9oOxgZ82r6SmCB45HVyL8tXm7mrOa7K8Y3r MFeV8fub/gtMEkQeqJdu4/vtgUv5m01ZY2cn3g6WSBTem7gDHp5ysWYqMltKeN5092t7M1bX QrWv1tjJijvZR11OQee5I8ZH9MSX1URfdcCEGJe30W92j5jBouWJxviDG0pF5IFaC2E3hen9 8S8zJ4b96jbkCzBV4A6vYQYUQBdDYx908ohwygH4PabmEhhm7lbjLwlhWideZFxcnG6MGLQv 8DYlWvdhN40OWiK3DnPuMyWDKwXRh4GkJpPdRlPo5TZj5h3OGBQ/5oTCeH/8c3oCORdYdWfC Op63gjxBUQ6StWMewbre5fttLD+VvV4pGsV7lDqIm/3fQimhW/loGvQsYRxeRanB8mvZbrrn qZ1NQveV/q44RtOx+WdIoJIhtqTdinr9o9tBtQnykKgT+0XmFXfHLdKOYvbzS8hZI7DjFdn2 meSTWMfW8u4mlg5J8xLAOim53t9ShHCP5SkcjjCIoBFJYApvjQDXSIERK0prGSBySm+TKbwl DFdLopodsSj9O8DOWlogn9eymnBdS9oyE6vjuAnNKVwqbxrqsRJpUwwCNDcDeA4fUK/48J4/ rLw6KvkLmccsotJzSmye3X2D1ocDgan/PnpKpu4bKzFXFiHjxA7pCLbRastvzuKsL8dfd4Wu l+8FXPSmS5TNYOqXVZP2tqmyUmfytObA37raggbmofXSYx9aZERAX42ebCG8Z2JZNEXTHdw4 x9M2DnqLlX2D+0zeSjJTY0PXgaKLuD5tTuz3V1IDIPb0i5b/ZCy1qMqaf9tI/spkZQsk1eOi wxT3RnFhdr+qE1YyZQdLXnR9irna6y2a5aDYypdwGRwIM+DiVLXwjGf9S/0lcT3roSjfEqpz o18YNV42pnCwGGBsp/vlJx4HICqjIh45aMTT4q7Uf3W2jHipPkXHsr4L/AA8EZrxzXYAjwls YA4S+UPeiAeYor4R0zD3bXIfgzdmRYTSqF7MmUMjIflx3ssnxSC5mkEY1SvnoMa4gaEERfNO khzdQQBf3mhK8+CM/Q1oJINOApw++IpGYIjuHe/i6Dgylp3WnV5cs1gnH8rrITLqFVXYdIlp 5UYwb0oT6OG1awDCDKS5oWupFefO3Q43I4ZTLKfNV7MgTDJCkmzIvrcojJlhuvGnzX5wFIAC EACOBOStVVMavxNSZaSxitg/xUkyZiIvlo8XOWQvnljLB/gruZi0LZ3SlH0jma4EW/4uC6kq vHkc7xFJpdMHkXBEEpXazagngIJ2UuJUPqRfRE1VD7aHQ/56A1FPM2W/snkD8JQ7QdJ6zQ0M hnQw23V8FZn6CtWluwHWfWJ8/oIQnAR2fSQ0RZH5zQRKBH/hOUvAcNjYz9MuNMU4jcycwYVs 3n2R7AF62u4YeuKIYbe1cjJC4uxG1dGn8dZL5cdYJibgdI+jKTxOj2vkTIZTax5LVpd/Wg9U Uy5hisFd6oBpGCP56IO7OMkct/n64Evlmian7EPV3p5kZMhUckousOCF4Rg6+6QjmZ3NTWon YeCgLzKZ66fbuou27BKTaJIlzkHZFv9kEZOahFEWcHkz/GkG3UvrY8d/y6p9GHfEV9KXp8MP 4B6/a3ZROILM77P3Z4Xe8KSQCDqTxnbsFnuQHqTIaX6mYvaeBEHA+9Sg3D6uSvQKnlV/vqSM Cb36F9uFawf2PRfKv6GKtFrAebysY+ga8l29yFdQPQlzMW/lFeiQ5UcGGKe1WAKCq9BqALkB t768wEMHIPpdYUXijBPEgJPmvgyGNIrBHOhRcIxZGaCuzrwLSD5GbnaYhOALmetWx/DTU9tW VRJWdPl8+Aej1NR6r6WSgXhxEuV/W7Vep6mK+oAA4GR1TzBi8tUUPSyqeZZVJr6VPsrMrPaQ jihlUtIAfOnLMFke1lU5QhomK4JQlz8aVhykbe1oGnTe8OZFuAYw0ldZPTkA8GPkWfvpvcWr yP7Mze2VnX4QzmnVHK/jcpL/rWLcAM2xZzY3seRxvjybsOsTL6Dkcf/Vi5oP5rH5sEXU1GzP ZesGFhrQcXJQhj33nI9aSE/hqqbe/YJfe76XuA2RWZXkHXtEZfvQN3SnCeOHZUAKnBNuLoeR w4MLC0FbaF1gpiustdU6WXC5SCykzSU8K4Sk7v1mp0FQA/9Tox/HTAuwJJnGAavcBRKuRTpC MqV9BORvHH/WYhi3bcvpAAAAd7UOZghsgm8AAZdDgu4DAGYAZBOxxGf7AgAAAAAEWVo= --3uo+9/B/ebqu+fSQ--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141212193447.GA1657>