Date: Sun, 30 Jul 2006 15:04:48 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: net@FreeBSD.org, arch@FreeBSD.org Subject: Changes in the network interface queueing handoff model Message-ID: <20060730141642.D16341@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1695162780-1154268288=:16341 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed 5BOne of the ideas that I, Scott Long, and a few others have been bouncing around for some time is a restructuring of the network interface packet transmission API to reduce the number of locking operations and allow network device drivers increased control of the queueing behavior. Right now, it works something like that following: - When a network protocol wants to transmit, it calls the ifnet's link layer output routine via ifp->if_output() with the ifnet pointer, packet, destination address information, and route information. - The link layer (e.g., ether_output() + ether_output_frame()) encapsulates the packet as necessary, performs a link layer address translation (such as ARP), and hands off to the ifnet driver via a call to IFQ_HANDOFF(), which accepts the ifnet pointer and packet. - The ifnet layer enqueues the packet in the ifnet send queue (ifp->if_snd), and then looks at the driver's IFF_DRV_OACTIVE flag to determine if it needs to "start" output by the driver. If the driver is already active, it doesn't, and otherwise, it does. - The driver dequeues the packet from ifp->if_snd, performs any driver encapsulation and wrapping, and notifies the hardware. In modern hardware, this consists of hooking the data of the packet up to the descriptor ring and notifying the hardware to pick it up via DMA. In order hardware, the driver would perform a series of I/O operations to send the entire packet directly to the card via a system bus. Why change this? A few reasons: - The ifnet layer send queue is becoming decreasingly useful over time. Most modern hardware has a significant number of slots in its transmit descriptor ring, tuned for the performance of the hardware, etc, which is the effective transmit queue in practice. The additional queue depth doesn't increase throughput substantially (if at all) but does consume memory. - On extremely fast hardware (with respect to CPU speed), the queue remains essentially empty, so we pay the cost of enqueueing and dequeuing a packet from an empty queue. - The ifnet send queue is a separately locked object from the device driver, meaning that for a single enqueue/dequeue pair, we pay an extra four lock operations (two for insert, two for remove) per packet. - For synthetic link layer drivers, such as if_vlan, which have no need for queueing at all, the cost of queueing is eliminated. - IFF_DRV_OACTIVE is no longer inspected by the link layer, only by the driver, which helps eliminate a latent race condition involving use of the flag. The proposed change is simple: right now one or more enqueue operations occurs, when a call to ifp->if_start() is made to notify the driver that it may need to do something (if the ACTIVE flag isn't set). In the new world order, the driver is directly passed the mbuf, and may then choose to queue it or otherwise handle it as it sees fit. The immediate practical benefit is clear: if the queueing at the ifnet layer is unnecessary, it is entirely avoided, skipping enqueue, dequeue, and four mutex operations. This applies immediately for VLAN processing, but also means that for modern gigabit cards, the hardware queue (which will be used anyway) is the only queue necessary. There are a few downsides, of course: - For older hardware without its own queueing, the queue is still required -- not only that, but we've now introduced an unconditional function pointer invocation, which on older hardware, is has more significant relative cost than it has on more recent CPUs. - If drivers still require or use a queue, they must now synchronize access to the queue. The obvious choices are to use the ifq lock (and restore the above four lock operations), or to use the driver mutex (and risk higher contention). Right now, if the driver is busy (driver mutex held) then an enqueue is still possible, but with this change and a single mutex protecting the send queue and driver, that is no longer possible. Attached is a patch that maintains the current if_start, but adds if_startmbuf. If a device driver implements if_startmbuf and the global sysctl net.startmbuf_enabled is set to 1, then the if_startmbuf path in the driver will be used. Otherwise, if_start is used. I have modified the if_em driver to implement if_startmbuf also. If there is no packet backlog in the if_snd queue, it directly places the packet in the transmit descriptor ring. If there is a backlog, it uses the if_snd queue protected by driver mutex, rather than a separate ifq mutex. In some basic local micro-benchmarks, I saw a 5% improvement in UDP 0-byte paylod PPS on UP, and a 10% improvement on SMP. I saw a 1.7% performance improvement in the bulk serving of 1k files over HTTP. These are only micro-benchmarks, and reflect a configuration in which the CPU is unable to keep up with the output rate of the 1gbps ethernet card in the device, so reductions in host CPU usage are immediately visible in increased output as the CPU is able to better keep up with the network hardware. Other configurations are also of interest of interesting, especially ones in which the network device is unable to keep up with the CPU, resulting in more queueing. Conceptual review as well as banchmarking, etc, would be most welcome. Robert N M Watson Computer Laboratory University of Cambridge --0-1695162780-1154268288=:16341 Content-Type: TEXT/plain; charset=US-ASCII; name=20060730-if_startmbuf.diff Content-Transfer-Encoding: BASE64 Content-ID: <20060730150448.G16341@fledge.watson.org> Content-Description: Content-Disposition: attachment; filename=20060730-if_startmbuf.diff LS0tIC8vZGVwb3QvdmVuZG9yL2ZyZWVic2Qvc3JjL3N5cy9kZXYvZW0vaWZf ZW0uYwkyMDA2LzA3LzI3IDAwOjQ2OjI0DQorKysgLy9kZXBvdC91c2VyL3J3 YXRzb24vaWZuZXQvc3JjL3N5cy9kZXYvZW0vaWZfZW0uYwkyMDA2LzA3LzI5 IDE4OjQzOjE0DQpAQCAtNzM1LDYgKzczNSw5NSBAQA0KIAlFTV9VTkxPQ0so c2MpOw0KIH0NCiANCitzdGF0aWMgaW50DQorZW1fc3RhcnRtYnVmKHN0cnVj dCBpZm5ldCAqaWZwLCBzdHJ1Y3QgbWJ1ZiAqbSkNCit7DQorICAgICAgICBz dHJ1Y3QgbWJ1ZiAgICAqbV9oZWFkOw0KKyAgICAgICAgc3RydWN0IGVtX3Nv ZnRjICpzYyA9IGlmcC0+aWZfc29mdGM7DQorCXN0cnVjdCBpZnF1ZXVlICpp ZnEgPSAoc3RydWN0IGlmcXVldWUgKikmaWZwLT5pZl9zbmQ7DQorDQorCS8q DQorCSAqIFRocmVlIGNhc2VzOg0KKwkgKg0KKwkgKiAoMSkgSW50ZXJmYWNl IGlzbid0IHJ1bm5pbmcsIGxpbmsgaXMgZG93biwgb3IgaXMgYWxyZWFkeSBh Y3RpdmUsDQorCSAqICAgICBldGMsIHNpbXBseSBlbnF1ZXVlLg0KKwkgKg0K KwkgKiAoMikgVGhlIGludGVyZmFjZSBpcyBydW5uaW5nLCBub3QgdG9vIGJ1 c3ksIGFuZCB3ZSBoYXZlIG5vIG1idWZzDQorCSAqICAgICBpbiB0aGUgaWZu ZXQgc2VuZCBxdWV1ZSwgc28gdHJ5IHRvIGhhbmQgZGlyZWN0bHkgdG8gaGFy ZHdhcmUuDQorCSAqDQorCSAqICgzKSBUaGUgaW50ZXJmYWNlIGlzIHJ1bm5p bmcsIGJ1dCB3ZSBoYXZlIGEgYmFja2xvZy4gIEluc2VydCB0aGUNCisJICog ICAgIGN1cnJlbnQgbWJ1ZiBpbnRvIHRoZSBxdWV1ZSBhbmQgcHJvY2VzcyBp bi1vcmRlciwgaWYgcG9zc2libGUuDQorCSAqLw0KKwlFTV9MT0NLKHNjKTsN CisJaWYgKCgoaWZwLT5pZl9kcnZfZmxhZ3MgJiAoSUZGX0RSVl9SVU5OSU5H fElGRl9EUlZfT0FDVElWRSkpICE9DQorCSAgICBJRkZfRFJWX1JVTk5JTkcp IHx8ICFzYy0+bGlua19hY3RpdmUpIHsNCisJCWlmIChfSUZfUUZVTEwoaWZx KSkgew0KKwkJCV9JRl9EUk9QKGlmcSk7DQorCQkJRU1fVU5MT0NLKHNjKTsN CisJCQltX2ZyZWVtKG0pOw0KKwkJCXJldHVybiAoRU5PQlVGUyk7DQorCQl9 DQorCQlfSUZfRU5RVUVVRShpZnEsIG0pOw0KKwkJRU1fVU5MT0NLKHNjKTsN CisJCXJldHVybiAoMCk7DQorCX0NCisNCisJLyoNCisJICogWFhYUlc6IFZh cmlvdXMgY2FzZXMgaGVyZSBoYXZlIGhpc3RvcmljYWxseSBjb3VudGVkIGFz IHN1Y2Nlc3NlcywNCisJICogYnV0IHBlcmhhcHMgdGhleSBzaG91bGQgcmV0 dXJuIEVOT0JVRlM/DQorCSAqLw0KKwlpZiAoX0lGX1FMRU4oaWZxKSA9PSAw KSB7DQorCSAJLyoNCisJCSAqIGVtX2VuY2FwKCkgY2FuIG1vZGlmeSBvdXIg cG9pbnRlciwgYW5kIG9yIG1ha2UgaXQgTlVMTCBvbg0KKwkJICogZmFpbHVy ZS4gIEluIHRoYXQgZXZlbnQsIHdlIGNhbid0IGVucXVldWUuDQorCQkgKi8N CisJCWlmIChlbV9lbmNhcChzYywgJm0pKSB7DQorCQkJaWYgKG0gPT0gTlVM TCkgew0KKwkJCQlFTV9VTkxPQ0soc2MpOw0KKwkJCQlyZXR1cm4gKDApOw0K KwkJCX0NCisJCQlpZnAtPmlmX2ZsYWdzIHw9IElGRl9EUlZfT0FDVElWRTsN CisJCQlfSUZfUFJFUEVORChpZnEsIG0pOw0KKwkJCUVNX1VOTE9DSyhzYyk7 DQorCQkJcmV0dXJuICgwKTsNCisJCX0NCisJCUJQRl9NVEFQKGlmcCwgbSk7 DQorCQlpZnAtPmlmX3RpbWVyID0gRU1fVFhfVElNRU9VVDsNCisJCUVNX1VO TE9DSyhzYyk7DQorCQlyZXR1cm4gKDApOw0KKwl9DQorDQorCWlmIChfSUZf UUZVTEwoaWZxKSkgew0KKwkJX0lGX0RST1AoaWZxKTsNCisJCUVNX1VOTE9D SyhzYyk7DQorCQltX2ZyZWVtKG0pOw0KKwkJcmV0dXJuIChFTk9CVUZTKTsN CisJfQ0KKwlfSUZfRU5RVUVVRShpZnEsIG0pOw0KKw0KKwl3aGlsZSAoIUlG UV9EUlZfSVNfRU1QVFkoJmlmcC0+aWZfc25kKSkgew0KKwkJSUZRX0RSVl9E RVFVRVVFKCZpZnAtPmlmX3NuZCwgbV9oZWFkKTsNCisJCWlmIChtX2hlYWQg PT0gTlVMTCkNCisJCQlicmVhazsNCisJIAkvKg0KKwkJICogZW1fZW5jYXAo KSBjYW4gbW9kaWZ5IG91ciBwb2ludGVyLCBhbmQgb3IgbWFrZSBpdCBOVUxM IG9uDQorCQkgKiBmYWlsdXJlLiAgSW4gdGhhdCBldmVudCwgd2UgY2FuJ3Qg cmVxdWV1ZS4NCisJCSAqLw0KKwkJaWYgKGVtX2VuY2FwKHNjLCAmbV9oZWFk KSkgew0KKwkJCWlmIChtX2hlYWQgPT0gTlVMTCkNCisJCQkJYnJlYWs7DQor CQkJaWZwLT5pZl9kcnZfZmxhZ3MgfD0gSUZGX0RSVl9PQUNUSVZFOw0KKwkJ CUlGUV9EUlZfUFJFUEVORCgmaWZwLT5pZl9zbmQsIG1faGVhZCk7DQorCQkJ YnJlYWs7DQorCQl9DQorCQlCUEZfTVRBUChpZnAsIG1faGVhZCk7DQorCQlp ZnAtPmlmX3RpbWVyID0gRU1fVFhfVElNRU9VVDsNCisJfQ0KKw0KKwlFTV9V TkxPQ0soc2MpOw0KKwlyZXR1cm4gKDApOw0KK30NCisNCiAvKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqDQogICogIElvY3RsIGVudHJ5IHBvaW50DQogICoN CkBAIC0yMTU0LDYgKzIyNDMsNyBAQA0KIAlpZnAtPmlmX2ZsYWdzID0gSUZG X0JST0FEQ0FTVCB8IElGRl9TSU1QTEVYIHwgSUZGX01VTFRJQ0FTVDsNCiAJ aWZwLT5pZl9pb2N0bCA9IGVtX2lvY3RsOw0KIAlpZnAtPmlmX3N0YXJ0ID0g ZW1fc3RhcnQ7DQorCWlmcC0+aWZfc3RhcnRtYnVmID0gZW1fc3RhcnRtYnVm Ow0KIAlpZnAtPmlmX3dhdGNoZG9nID0gZW1fd2F0Y2hkb2c7DQogCUlGUV9T RVRfTUFYTEVOKCZpZnAtPmlmX3NuZCwgc2MtPm51bV90eF9kZXNjIC0gMSk7 DQogCWlmcC0+aWZfc25kLmlmcV9kcnZfbWF4bGVuID0gc2MtPm51bV90eF9k ZXNjIC0gMTsNCi0tLSAvL2RlcG90L3ZlbmRvci9mcmVlYnNkL3NyYy9zeXMv bmV0L2lmLmMJMjAwNi8wNy8wOSAwNjowNjoyNQ0KKysrIC8vZGVwb3QvdXNl ci9yd2F0c29uL2lmbmV0L3NyYy9zeXMvbmV0L2lmLmMJMjAwNi8wNy8yNiAx NzozMjo1MA0KQEAgLTI0ODYsMjggKzI0ODYsMTExIEBADQogCShpZnAtPmlm X3N0YXJ0KShpZnApOw0KIH0NCiANCitzdGF0aWMgaW50CXN0YXJ0bWJ1Zl9l bmFibGVkOw0KK1NZU0NUTF9JTlQoX25ldCwgT0lEX0FVVE8sIHN0YXJ0bWJ1 Zl9lbmFibGVkLCBDVExGTEFHX1JXLCAmc3RhcnRtYnVmX2VuYWJsZWQsDQor ICAgIDAsICIiKTsNCisNCisvKg0KKyAqIFhYWFJXOg0KKyAqDQorICogaWZf dmFyLmggYW5kIHRoZSBpbnRlcmZhY2UgaGFuZG9mZiBhcmUgc29tZSBvZiB0 aGUgbmFzdGllc3QgcGllY2VzIG9mIHRoZQ0KKyAqIEJTRCBuZXR3b3JrIHN0 YWNrLiAgR2VuZXJhdGlvbnMgb2YgaGFja3MsIHZhcmlhbnRzLCBpbmNvbnNp c3RlbmN5LCBhbmQNCisgKiBmb29saXNobmVzcyBoYXZlIHJlc3VsdGVkIGlu IGVzc2VudGlhbGx5IHVucmVhZGFibGUgY29kZS4gIEZvciBleGFtcGxlLA0K KyAqIHdoeSBhcmUgdGhlIGlmcV8qIGludGVyZmFjZXMgdGhlIG9uZXMgdGhh dCB1c2UgdGhlIGRlZmF1bHQgaWZuZXQgc2VuZA0KKyAqIHF1ZXVlLCBhbmQg dGhlIGlmXyogaW50ZXJmYWNlcyB0aGUgb25lcyB0aGF0IHVzZSBhbHRlcm5h dGl2ZSBxdWV1ZXMsDQorICogcG9zc2libHkgd2l0aCBubyBpZm5ldCBhdCBh bGw/ICBBbmQgd2h5IGRvIHNvbWUgaW50ZXJmYWNlcyByZXR1cm4gZXJybm8N CisgKiB2YWx1ZXMsIGJ1dCBvdGhlcnMgYm9vbGVhbnM/DQorICovDQorDQor LyoNCisgKiBIYW5kb2ZmIGZ1bmN0aW9uIGZvciBzaW1wbGUgaWZuZXQgc3Ry dWN0dXJlcy4gIFJldHVybnMgYW4gZXJybm8gdmFsdWUuDQorICovDQogaW50 DQotaWZfaGFuZG9mZihzdHJ1Y3QgaWZxdWV1ZSAqaWZxLCBzdHJ1Y3QgbWJ1 ZiAqbSwgc3RydWN0IGlmbmV0ICppZnAsIGludCBhZGp1c3QpDQoraWZxX2hh bmRvZmYoc3RydWN0IGlmbmV0ICppZnAsIHN0cnVjdCBtYnVmICptLCBpbnQg YWRqdXN0KQ0KK3sNCisJaW50IGVycm9yLCBsZW4sIHN0YXJ0bWJ1ZjsNCisJ c2hvcnQgbWZsYWdzOw0KKw0KKwlsZW4gPSBtLT5tX3BrdGhkci5sZW47DQor CW1mbGFncyA9IG0tPm1fZmxhZ3M7DQorDQorCWlmIChzdGFydG1idWZfZW5h YmxlZCAmJiBpZnAtPmlmX3N0YXJ0bWJ1ZiAhPSBOVUxMKQ0KKwkJc3RhcnRt YnVmID0gMTsNCisJZWxzZQ0KKwkJc3RhcnRtYnVmID0gMDsNCisNCisJaWYg KHN0YXJ0bWJ1ZikNCisJCWVycm9yID0gaWZwLT5pZl9zdGFydG1idWYoaWZw LCBtKTsNCisJZWxzZQ0KKwkJSUZRX0VOUVVFVUUoJmlmcC0+aWZfc25kLCBt LCBlcnJvcik7DQorCWlmIChlcnJvciA9PSAwKSB7DQorCQlpZnAtPmlmX29i eXRlcyArPSBsZW4gKyBhZGp1c3Q7DQorCQlpZiAobWZsYWdzICYgKE1fQkNB U1R8TV9NQ0FTVCkpDQorCQkJaWZwLT5pZl9vbWNhc3RzKys7DQorCX0NCisJ aWYgKCFzdGFydG1idWYgJiYgKGlmcC0+aWZfZHJ2X2ZsYWdzICYgSUZGX0RS Vl9PQUNUSVZFKSA9PSAwKQ0KKwkJaWZfc3RhcnQoaWZwKTsNCisJcmV0dXJu IChlcnJvcik7DQorfQ0KKw0KKy8qDQorICogSGFuZG9mZiBmdW5jdGlvbiBm b3IgYW4gaWZxdWV1ZSB3aXRoIGFuIG9wdGlvbmFsbHkgYWZmaWxpdGlhdGVk IGlmbmV0Lg0KKyAqIFJldHVybnMgYSBib29sZWFuLg0KKyAqLw0KK2ludA0K K2lmX2hhbmRvZmYoc3RydWN0IGlmcXVldWUgKmlmcSwgc3RydWN0IG1idWYg Km0sIHN0cnVjdCBpZm5ldCAqaWZwLA0KKyAgICBpbnQgYWRqdXN0KQ0KK3sN CisJaW50IGxlbiwgYWN0aXZlLCBzdGFydG1idWYsIHN1Y2Nlc3M7DQorCXNo b3J0IG1mbGFnczsNCisNCisJYWN0aXZlID0gMDsNCisJbGVuID0gbS0+bV9w a3RoZHIubGVuOw0KKwltZmxhZ3MgPSBtLT5tX2ZsYWdzOw0KKw0KKwlpZiAo c3RhcnRtYnVmX2VuYWJsZWQgJiYgaWZwICE9IE5VTEwgJiYgaWZwLT5pZl9z dGFydG1idWYgIT0gTlVMTCkNCisJCXN0YXJ0bWJ1ZiA9IDE7DQorCWVsc2UN CisJCXN0YXJ0bWJ1ZiA9IDA7DQorDQorCWlmIChzdGFydG1idWYpDQorCQlz dWNjZXNzID0gKGlmcC0+aWZfc3RhcnRtYnVmKGlmcCwgbSkgPT0gMCk7DQor CWVsc2Ugew0KKwkJSUZfTE9DSyhpZnEpOw0KKwkJaWYgKF9JRl9RRlVMTChp ZnEpKSB7DQorCQkJX0lGX0RST1AoaWZxKTsNCisJCQltX2ZyZWVtKG0pOw0K KwkJCXN1Y2Nlc3MgPSAwOw0KKwkJfSBlbHNlIHsNCisJCQlfSUZfRU5RVUVV RShpZnEsIG0pOw0KKwkJCXN1Y2Nlc3MgPSAxOw0KKwkJfQ0KKwkJSUZfVU5M T0NLKGlmcSk7DQorCQlpZiAoaWZwICE9IE5VTEwgJiYgIShpZnAtPmlmX2Ry dl9mbGFncyAmIElGRl9EUlZfT0FDVElWRSkpDQorCQkJaWZfc3RhcnQoaWZw KTsNCisJfQ0KKwlpZiAoc3VjY2VzcyAmJiBpZnAgIT0gTlVMTCkgew0KKwkJ aWZwLT5pZl9vYnl0ZXMgKz0gbGVuICsgYWRqdXN0Ow0KKwkJaWYgKG0tPm1f ZmxhZ3MgJiAoTV9CQ0FTVHxNX01DQVNUKSkNCisJCQlpZnAtPmlmX29tY2Fz dHMrKzsNCisJfQ0KKwlyZXR1cm4gKHN1Y2Nlc3MpOw0KK30NCisNCisvKg0K KyAqIFV0aWxpdHkgZnVuY3Rpb24gdG8gYmUgdXNlZCBieSBkZXZpY2UgZHJp dmVycyB3aGVuIHRoZXkgbmVlZCB0byBlbnF1ZXVlIGENCisgKiBwYWNrZXQg dG8gYW4gaW50ZXJmYWNlLXJlbGF0ZWQgcXVldWUgcmF0aGVyIHRoYW4gaW1t ZWRpYXRlbHkgZGVsaXZlcmluZy4NCisgKi8NCitpbnQNCitpZl9zdGFydG1i dWZfZW5xdWV1ZShzdHJ1Y3QgaWZxdWV1ZSAqaWZxLCBzdHJ1Y3QgbWJ1ZiAq bSkNCiB7DQotCWludCBhY3RpdmUgPSAwOw0KIA0KLQlJRl9MT0NLKGlmcSk7 DQogCWlmIChfSUZfUUZVTEwoaWZxKSkgew0KIAkJX0lGX0RST1AoaWZxKTsN Ci0JCUlGX1VOTE9DSyhpZnEpOw0KIAkJbV9mcmVlbShtKTsNCiAJCXJldHVy biAoMCk7DQogCX0NCi0JaWYgKGlmcCAhPSBOVUxMKSB7DQotCQlpZnAtPmlm X29ieXRlcyArPSBtLT5tX3BrdGhkci5sZW4gKyBhZGp1c3Q7DQotCQlpZiAo bS0+bV9mbGFncyAmIChNX0JDQVNUfE1fTUNBU1QpKQ0KLQkJCWlmcC0+aWZf b21jYXN0cysrOw0KLQkJYWN0aXZlID0gaWZwLT5pZl9kcnZfZmxhZ3MgJiBJ RkZfRFJWX09BQ1RJVkU7DQotCX0NCiAJX0lGX0VOUVVFVUUoaWZxLCBtKTsN Ci0JSUZfVU5MT0NLKGlmcSk7DQotCWlmIChpZnAgIT0gTlVMTCAmJiAhYWN0 aXZlKQ0KLQkJaWZfc3RhcnQoaWZwKTsNCiAJcmV0dXJuICgxKTsNCiB9DQog DQotLS0gLy9kZXBvdC92ZW5kb3IvZnJlZWJzZC9zcmMvc3lzL25ldC9pZl92 YXIuaAkyMDA2LzA2LzE5IDIyOjIxOjIyDQorKysgLy9kZXBvdC91c2VyL3J3 YXRzb24vaWZuZXQvc3JjL3N5cy9uZXQvaWZfdmFyLmgJMjAwNi8wNy8zMCAx MDoxMTo1NA0KQEAgLTE2Miw3ICsxNjIsOCBAQA0KIAkJKHN0cnVjdCBpZm5l dCAqLCBzdHJ1Y3Qgc29ja2FkZHIgKiosIHN0cnVjdCBzb2NrYWRkciAqKTsN CiAJc3RydWN0CWlmYWRkcgkqaWZfYWRkcjsJLyogcG9pbnRlciB0byBsaW5r LWxldmVsIGFkZHJlc3MgKi8NCiAJdm9pZAkqaWZfc3BhcmUyOwkJLyogc3Bh cmUgcG9pbnRlciAyICovDQotCXZvaWQJKmlmX3NwYXJlMzsJCS8qIHNwYXJl IHBvaW50ZXIgMyAqLw0KKwlpbnQJKCppZl9zdGFydG1idWYpCQkvKiBlbnF1 ZXVlIGFuZCBzdGFydCBvdXRwdXQgKi8NCisJCShzdHJ1Y3QgaWZuZXQgKiwg c3RydWN0IG1idWYgKik7DQogCWludAlpZl9kcnZfZmxhZ3M7CQkvKiBkcml2 ZXItbWFuYWdlZCBzdGF0dXMgZmxhZ3MgKi8NCiAJdV9pbnQJaWZfc3BhcmVf ZmxhZ3MyOwkvKiBzcGFyZSBmbGFncyAyICovDQogCXN0cnVjdCAgaWZhbHRx IGlmX3NuZDsJCS8qIG91dHB1dCBxdWV1ZSAoaW5jbHVkZXMgYWx0cSkgKi8N CkBAIC0zNzAsMTIgKzM3MSwxNSBAQA0KIAkJbXR4X3VubG9jaygmR2lhbnQp OwkJCQkJXA0KIH0gd2hpbGUgKDApDQogDQoraW50CWlmcV9oYW5kb2ZmKHN0 cnVjdCBpZm5ldCAqaWZwLCBzdHJ1Y3QgbWJ1ZiAqbSwgaW50IGFkanVzdCk7 DQogaW50CWlmX2hhbmRvZmYoc3RydWN0IGlmcXVldWUgKmlmcSwgc3RydWN0 IG1idWYgKm0sIHN0cnVjdCBpZm5ldCAqaWZwLA0KIAkgICAgaW50IGFkanVz dCk7DQoraW50CWlmX3N0YXJ0bWJ1Zl9lbnF1ZXVlKHN0cnVjdCBpZnF1ZXVl ICppZnEsIHN0cnVjdCBtYnVmICptKTsNCisNCisjZGVmaW5lCUlGX0hBTkRP RkZfQURKKGlmcSwgbSwgaWZwLCBhZGopCVwNCisJaWZfaGFuZG9mZigoc3Ry dWN0IGlmcXVldWUgKilpZnEsIG0sIGlmcCwgYWRqKQ0KICNkZWZpbmUJSUZf SEFORE9GRihpZnEsIG0sIGlmcCkJCQlcDQogCWlmX2hhbmRvZmYoKHN0cnVj dCBpZnF1ZXVlICopaWZxLCBtLCBpZnAsIDApDQotI2RlZmluZQlJRl9IQU5E T0ZGX0FESihpZnEsIG0sIGlmcCwgYWRqKQlcDQotCWlmX2hhbmRvZmYoKHN0 cnVjdCBpZnF1ZXVlICopaWZxLCBtLCBpZnAsIGFkaikNCiANCiB2b2lkCWlm X3N0YXJ0KHN0cnVjdCBpZm5ldCAqKTsNCiANCkBAIC00NTksMjUgKzQ2Myw4 IEBADQogI2RlZmluZQlJRlFfSU5DX0RST1BTKGlmcSkJCSgoaWZxKS0+aWZx X2Ryb3BzKyspDQogI2RlZmluZQlJRlFfU0VUX01BWExFTihpZnEsIGxlbikJ KChpZnEpLT5pZnFfbWF4bGVuID0gKGxlbikpDQogDQotLyoNCi0gKiBUaGUg SUZGX0RSVl9PQUNUSVZFIHRlc3Qgc2hvdWxkIHJlYWxseSBvY2N1ciBpbiB0 aGUgZGV2aWNlIGRyaXZlciwgbm90IGluDQotICogdGhlIGhhbmRvZmYgbG9n aWMsIGFzIHRoYXQgZmxhZyBpcyBsb2NrZWQgYnkgdGhlIGRldmljZSBkcml2 ZXIuDQotICovDQotI2RlZmluZQlJRlFfSEFORE9GRl9BREooaWZwLCBtLCBh ZGosIGVycikJCQkJXA0KLWRvIHsJCQkJCQkJCQlcDQotCWludCBsZW47CQkJ CQkJCVwNCi0Jc2hvcnQgbWZsYWdzOwkJCQkJCQlcDQotCQkJCQkJCQkJXA0K LQlsZW4gPSAobSktPm1fcGt0aGRyLmxlbjsJCQkJCVwNCi0JbWZsYWdzID0g KG0pLT5tX2ZsYWdzOwkJCQkJCVwNCi0JSUZRX0VOUVVFVUUoJihpZnApLT5p Zl9zbmQsIG0sIGVycik7CQkJCVwNCi0JaWYgKChlcnIpID09IDApIHsJCQkJ CQlcDQotCQkoaWZwKS0+aWZfb2J5dGVzICs9IGxlbiArIChhZGopOwkJCVwN Ci0JCWlmIChtZmxhZ3MgJiBNX01DQVNUKQkJCQkJXA0KLQkJCShpZnApLT5p Zl9vbWNhc3RzKys7CQkJCVwNCi0JCWlmICgoKGlmcCktPmlmX2Rydl9mbGFn cyAmIElGRl9EUlZfT0FDVElWRSkgPT0gMCkJXA0KLQkJCWlmX3N0YXJ0KGlm cCk7CQkJCQlcDQotCX0JCQkJCQkJCVwNCisjZGVmaW5lCUlGUV9IQU5ET0ZG X0FESihpZnAsIG0sIGFkaiwgZXJyKSBkbyB7CQkJCVwNCisJZXJyID0gaWZx X2hhbmRvZmYoaWZwLCBtLCBhZGopOwkJCQkJXA0KIH0gd2hpbGUgKDApDQog DQogI2RlZmluZQlJRlFfSEFORE9GRihpZnAsIG0sIGVycikJCQkJCVwNCg== --0-1695162780-1154268288=:16341--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060730141642.D16341>