Date: Wed, 12 Sep 2007 17:21:13 -0400 From: "Robert Wojciechowski" <robertw@expressyard.com> To: <pyunyh@gmail.com> Cc: shigeaki@f.csce.kyushu-u.ac.jp, freebsd-net@freebsd.org, Josh Mouch <jmouch@expressyard.com> Subject: RE: FreeBSD nfe driver and IPMI cards Message-ID: <85D4F2C294E8434CA0AF775741532686623694@server1.ssgi.local> In-Reply-To: <20070912004554.GA8992@cdnetworks.co.kr> References: <85D4F2C294E8434CA0AF775741532686623679@server1.ssgi.local> <20070912004554.GA8992@cdnetworks.co.kr>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. ------_=_NextPart_001_01C7F582.C3A05A33 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable > > I'm the FreeBSD nfe driver from > > http://www.f.csce.kyushu-u.ac.jp/~shigeaki/software/freebsd-nfe.html > > with FreeBSD 6-stable with good results for the most part. The only > > issue I've experienced is that during a detach/shutdown of if_nfe, > the > > IPMI IP address I have set on my servers ceases to respond as well > as > > the ability to manage the servers. > > > > > > > > I traced the problem down to nfe_stop() and the fact that it > completely > > disables the Rx and Tx on the NIC. I have patched the driver to not > > disable the Rx/Tx and IPMI continues to work after a 'ifconfig nfe0 > > down', 'shutdown -p now', etc. > > > > > > > > Does anyone have any comments on this change I've made and any > possible > > side effects? Can this be included in the mainstream distribution of > the >=20 > Because MAC is still alive if's possible to recieve a packet. All DMA > maps are unloaded and buffers are already freed in nfe_stop so it > would cause panic I guess. But I'm not familiar with IPMI so I'm not > sure. Interesting, that is an issue that was also resolved by the forcedeth driver in Linux by resetting in nv_close to prevent DMA into freed memory. >=20 > > nfe drivers (and updated in 7-CURRENT) without causing any adverse > > problems? > > >=20 > I have no experience on IPMI but the change you've made would not > completely solve the issue. I guess supporting IPMI needs lots of > more work including: > o Autodetect IPMI capability. > o Autodetect active IPMI session in device attach and don't blindly > reset MAC/PHY. > o Don't blindly stop Tx/Rx on device detach. > Given that lack of publicly available datasheet for the hardware > supporing IPMI would be severly limited. Fortunately Linux seems to > have basic IPMI support in their forcedeth driver. Their code doesn't > easy to read but you may see what should be done in driver. However > I have no idea what we can do when active IPMI session is present in > driver attach phase. Normally PHY driver would reset PHY hardware > itself in driver attach which in turn would result in losing the IPMI > connection. >=20 Since we have no idea how to auto-detect IPMI, I added a device sysctl (enable_ipmi) to control the behavior I'm testing in-house right now. If the enable_ipmi sysctl is enabled, it'll: * resets after disabling the Rx/Tx and before disabling interrupts * re-enables the Rx/Tx after the ring buffers are freed, just like forcedeth but also enabling the Tx for IPMI (Linux just enabled the Rx for WOL) I have attached my new patch to this email that does the above, guarded by the sysctl. I have no idea how to handle the second case you mentioned, during attach. It does indeed cause a disruption in IPMI, but only for a few seconds. Where is the MAC and PHY reset and if it wasn't reset when using IPMI, what problems could it cause? -- Robert ------_=_NextPart_001_01C7F582.C3A05A33 Content-Type: application/octet-stream; name="nfe-ipmi-2.patch" Content-Transfer-Encoding: base64 Content-Description: nfe-ipmi-2.patch Content-Disposition: attachment; filename="nfe-ipmi-2.patch" ZGlmZiAtdXIgbmZlLTIwMDcwODE2L2lmX25mZS5jIG5mZS0yMDA3MDgxNi1zc2dpL2lmX25mZS5j Ci0tLSBuZmUtMjAwNzA4MTYvaWZfbmZlLmMJVHVlIFNlcCAxMSAwNzozNjoyNCAyMDA3CisrKyBu ZmUtMjAwNzA4MTYtc3NnaS9pZl9uZmUuYwlXZWQgU2VwIDEyIDEzOjAyOjM3IDIwMDcKQEAgLTI0 LDcgKzI0LDcgQEAKIF9fRkJTRElEKCIkRnJlZUJTRDogc3JjL3N5cy9kZXYvbmZlL2lmX25mZS5j LHYgMS4xOSAyMDA3LzA3LzI0IDAxOjExOjAwIHlvbmdhcmkgRXhwICQiKTsKIAogLyogVW5jb21t ZW50IHRoZSBmb2xsb3dpbmcgbGluZSB0byBlbmFibGUgcG9sbGluZy4gKi8KLSNkZWZpbmUJREVW SUNFX1BPTExJTkcKKy8qICNkZWZpbmUJREVWSUNFX1BPTExJTkcgKi8KIAogI2lmZGVmIEhBVkVf S0VSTkVMX09QVElPTl9IRUFERVJTCiAjaW5jbHVkZSAib3B0X2RldmljZV9wb2xsaW5nLmgiCkBA IC04MSw2ICs4MSw3IEBACiBzdGF0aWMgaW50ICBuZmVfcmVzdW1lKGRldmljZV90KTsKIHN0YXRp YyB2b2lkIG5mZV9zaHV0ZG93bihkZXZpY2VfdCk7CiBzdGF0aWMgdm9pZCBuZmVfcG93ZXIoc3Ry dWN0IG5mZV9zb2Z0YyAqKTsKK3N0YXRpYyB2b2lkIG5mZV9yZXNldChzdHJ1Y3QgbmZlX3NvZnRj ICopOwogc3RhdGljIGludCAgbmZlX21paWJ1c19yZWFkcmVnKGRldmljZV90LCBpbnQsIGludCk7 CiBzdGF0aWMgaW50ICBuZmVfbWlpYnVzX3dyaXRlcmVnKGRldmljZV90LCBpbnQsIGludCwgaW50 KTsKIHN0YXRpYyB2b2lkIG5mZV9taWlidXNfc3RhdGNoZyhkZXZpY2VfdCk7CkBAIC0xMjcsNiAr MTI4LDcgQEAKIHN0YXRpYyB2b2lkIG5mZV9kbWFfbWFwX3NlZ3Modm9pZCAqLCBidXNfZG1hX3Nl Z21lbnRfdCAqLCBpbnQsIGludCk7CiAKIHN0YXRpYyBpbnQgc3lzY3RsX2ludF9yYW5nZShTWVND VExfSEFORExFUl9BUkdTLCBpbnQsIGludCk7CitzdGF0aWMgaW50IHN5c2N0bF9od19uZmVfZW5h YmxlX2lwbWkoU1lTQ1RMX0hBTkRMRVJfQVJHUyk7CiBzdGF0aWMgaW50IHN5c2N0bF9od19uZmVf cHJvY19saW1pdChTWVNDVExfSEFORExFUl9BUkdTKTsKIAogI2lmZGVmIE5GRV9ERUJVRwpAQCAt NTM0LDYgKzUzNiwyMyBAQAogCiAJU1lTQ1RMX0FERF9QUk9DKGRldmljZV9nZXRfc3lzY3RsX2N0 eChkZXYpLAogCSAgICBTWVNDVExfQ0hJTERSRU4oZGV2aWNlX2dldF9zeXNjdGxfdHJlZShkZXYp KSwKKwkgICAgT0lEX0FVVE8sICJlbmFibGVfaXBtaSIsIENUTFRZUEVfSU5UIHwgQ1RMRkxBR19S VywKKwkgICAgJnNjLT5uZmVfZW5hYmxlX2lwbWksIDAsIHN5c2N0bF9od19uZmVfZW5hYmxlX2lw bWksICJJIiwKKwkgICAgImVuYWJsZSBJUE1JIik7CisKKwlzYy0+bmZlX2VuYWJsZV9pcG1pID0g TkZFX0VOQUJMRV9JUE1JX0RFRkFVTFQ7CisJZXJyb3IgPSByZXNvdXJjZV9pbnRfdmFsdWUoZGV2 aWNlX2dldF9uYW1lKGRldiksIGRldmljZV9nZXRfdW5pdChkZXYpLAorCSAgICAiZW5hYmxlX2lw bWkiLCAmc2MtPm5mZV9lbmFibGVfaXBtaSk7CisJaWYgKGVycm9yID09IDApIHsKKwkJaWYgKHNj LT5uZmVfcHJvY2Vzc19saW1pdCA8IDAgfHwgc2MtPm5mZV9wcm9jZXNzX2xpbWl0ID4gMSkgewor CQkJZGV2aWNlX3ByaW50ZihkZXYsICJlbmFibGVfaXBtaSB2YWx1ZSBvdXQgb2YgcmFuZ2U7ICIK KwkJCSAgICAidXNpbmcgZGVmYXVsdDogJWRcbiIsIE5GRV9FTkFCTEVfSVBNSV9ERUZBVUxUKTsK KwkJCXNjLT5uZmVfZW5hYmxlX2lwbWkgPSBORkVfRU5BQkxFX0lQTUlfREVGQVVMVDsKKwkJfQor CX0KKworCVNZU0NUTF9BRERfUFJPQyhkZXZpY2VfZ2V0X3N5c2N0bF9jdHgoZGV2KSwKKwkgICAg U1lTQ1RMX0NISUxEUkVOKGRldmljZV9nZXRfc3lzY3RsX3RyZWUoZGV2KSksCiAJICAgIE9JRF9B VVRPLCAicHJvY2Vzc19saW1pdCIsIENUTFRZUEVfSU5UIHwgQ1RMRkxBR19SVywKIAkgICAgJnNj LT5uZmVfcHJvY2Vzc19saW1pdCwgMCwgc3lzY3RsX2h3X25mZV9wcm9jX2xpbWl0LCAiSSIsCiAJ ICAgICJtYXggbnVtYmVyIG9mIFJ4IGV2ZW50cyB0byBwcm9jZXNzIik7CkBAIC03NzksMTIgKzc5 OCw3IEBACiAKIAlpZiAoKHNjLT5uZmVfZmxhZ3MgJiBORkVfUFdSX01HTVQpID09IDApCiAJCXJl dHVybjsKLQlORkVfV1JJVEUoc2MsIE5GRV9SWFRYX0NUTCwgTkZFX1JYVFhfUkVTRVQgfCBORkVf UlhUWF9CSVQyKTsKLQlORkVfV1JJVEUoc2MsIE5GRV9NQUNfUkVTRVQsIE5GRV9NQUNfUkVTRVRf TUFHSUMpOwotCURFTEFZKDEwMCk7Ci0JTkZFX1dSSVRFKHNjLCBORkVfTUFDX1JFU0VULCAwKTsK LQlERUxBWSgxMDApOwotCU5GRV9XUklURShzYywgTkZFX1JYVFhfQ1RMLCBORkVfUlhUWF9CSVQy KTsKKwluZmVfcmVzZXQoc2MpOwogCXB3ciA9IE5GRV9SRUFEKHNjLCBORkVfUFdSMl9DVEwpOwog CXB3ciAmPSB+TkZFX1BXUjJfV0FLRVVQX01BU0s7CiAJaWYgKHNjLT5uZmVfcmV2aWQgPj0gMHhh MyAmJgpAQCAtNzk0LDYgKzgwOCwxNSBAQAogCU5GRV9XUklURShzYywgTkZFX1BXUjJfQ1RMLCBw d3IpOwogfQogCitzdGF0aWMgdm9pZAorbmZlX3Jlc2V0KHN0cnVjdCBuZmVfc29mdGMgKnNjKQor eworCU5GRV9XUklURShzYywgTkZFX1JYVFhfQ1RMLCBORkVfUlhUWF9SRVNFVCB8IE5GRV9SWFRY X0JJVDIpOworCU5GRV9XUklURShzYywgTkZFX01BQ19SRVNFVCwgTkZFX01BQ19SRVNFVF9NQUdJ Qyk7CisJREVMQVkoMTAwKTsKKwlORkVfV1JJVEUoc2MsIE5GRV9NQUNfUkVTRVQsIDApOworCURF TEFZKDEwMCk7Cit9CiAKIHN0YXRpYyB2b2lkCiBuZmVfbWlpYnVzX3N0YXRjaGcoZGV2aWNlX3Qg ZGV2KQpAQCAtMzAwOSw2ICszMDMyLDExIEBACiAJLyogZGlzYWJsZSBSeCAqLwogCU5GRV9XUklU RShzYywgTkZFX1JYX0NUTCwgMCk7CiAKKwkvKiByZXNldCB0aGUgVHgvUnggc28gdGhlIE5JQyBk b2VzIG5vdCBETUEgaW50byBmcmVlZCBtZW1vcnkgKi8KKwlpZiAoJnNjLT5uZmVfZW5hYmxlX2lw bWkpIHsKKwkJbmZlX3Jlc2V0KHNjKTsKKwl9CisKIAkvKiBkaXNhYmxlIGludGVycnVwdHMgKi8K IAluZmVfZGlzYWJsZV9pbnRyKHNjKTsKIApAQCAtMzA1NSw2ICszMDgzLDEyIEBACiAJCQl0ZGF0 YS0+bSA9IE5VTEw7CiAJCX0KIAl9CisKKwkvKiBlbmFibGUgdGhlIFR4L1J4IGZvciBJUE1JIHN1 cHBvcnQgKi8KKwlpZiAoJnNjLT5uZmVfZW5hYmxlX2lwbWkpIHsKKwkJTkZFX1dSSVRFKHNjLCBO RkVfVFhfQ1RMLCBORkVfVFhfU1RBUlQpOworCQlORkVfV1JJVEUoc2MsIE5GRV9SWF9DVEwsIE5G RV9SWF9TVEFSVCk7CisJfQogfQogCiAKQEAgLTMyMTAsNCArMzI0NCwxMSBAQAogCiAJcmV0dXJu IChzeXNjdGxfaW50X3JhbmdlKG9pZHAsIGFyZzEsIGFyZzIsIHJlcSwgTkZFX1BST0NfTUlOLAog CSAgICBORkVfUFJPQ19NQVgpKTsKK30KKworCitzdGF0aWMgaW50CitzeXNjdGxfaHdfbmZlX2Vu YWJsZV9pcG1pKFNZU0NUTF9IQU5ETEVSX0FSR1MpCit7CisJcmV0dXJuIChzeXNjdGxfaW50X3Jh bmdlKG9pZHAsIGFyZzEsIGFyZzIsIHJlcSwgMCwgMSkpOwogfQpPbmx5IGluIG5mZS0yMDA3MDgx Ni1zc2dpOiBpZl9uZmUuYy5vcmlnCk9ubHkgaW4gbmZlLTIwMDcwODE2LXNzZ2k6IGlmX25mZS5j LnJlagpCaW5hcnkgZmlsZXMgbmZlLTIwMDcwODE2L2lmX25mZS5rbyBhbmQgbmZlLTIwMDcwODE2 LXNzZ2kvaWZfbmZlLmtvIGRpZmZlcgpCaW5hcnkgZmlsZXMgbmZlLTIwMDcwODE2L2lmX25mZS5v IGFuZCBuZmUtMjAwNzA4MTYtc3NnaS9pZl9uZmUubyBkaWZmZXIKZGlmZiAtdXIgbmZlLTIwMDcw ODE2L2lmX25mZXJlZy5oIG5mZS0yMDA3MDgxNi1zc2dpL2lmX25mZXJlZy5oCi0tLSBuZmUtMjAw NzA4MTYvaWZfbmZlcmVnLmgJV2VkIEF1ZyAxNSAyMDo1MToxOCAyMDA3CisrKyBuZmUtMjAwNzA4 MTYtc3NnaS9pZl9uZmVyZWcuaAlXZWQgU2VwIDEyIDEyOjQ1OjM3IDIwMDcKQEAgLTIyLDYgKzIy LDcgQEAKICNkZWZpbmUJTkZFX0pVTUJPX1JYX1JJTkdfQ09VTlQJTkZFX1JYX1JJTkdfQ09VTlQK ICNkZWZpbmUJTkZFX1RYX1JJTkdfQ09VTlQJMjU2CiAKKyNkZWZpbmUJTkZFX0VOQUJMRV9JUE1J X0RFRkFVTFQJMAogI2RlZmluZQlORkVfUFJPQ19ERUZBVUxUCSgoTkZFX1JYX1JJTkdfQ09VTlQg KiAzKSAvIDQpCiAjZGVmaW5lCU5GRV9QUk9DX01JTgkJNTAKICNkZWZpbmUJTkZFX1BST0NfTUFY CQkoTkZFX1JYX1JJTkdfQ09VTlQgLSAxKQpkaWZmIC11ciBuZmUtMjAwNzA4MTYvaWZfbmZldmFy LmggbmZlLTIwMDcwODE2LXNzZ2kvaWZfbmZldmFyLmgKLS0tIG5mZS0yMDA3MDgxNi9pZl9uZmV2 YXIuaAlXZWQgQXVnIDE1IDIwOjUxOjE4IDIwMDcKKysrIG5mZS0yMDA3MDgxNi1zc2dpL2lmX25m ZXZhci5oCVdlZCBTZXAgMTIgMTI6MzU6NDcgMjAwNwpAQCAtMTE1LDYgKzExNSw3IEBACiAJaW50 CQkJbmZlX3N1c3BlbmRlZDsKIAlpbnQJCQluZmVfZnJhbWVzaXplOwogCWludAkJCW5mZV9wcm9j ZXNzX2xpbWl0OworCWludAkJCW5mZV9lbmFibGVfaXBtaTsKIAlpbnQJCQluZmVfZm9yY2VfdHg7 CiAJdWludDMyX3QJCW5mZV9pcnFfc3RhdHVzOwogCXVpbnQzMl90CQluZmVfaXJxX21hc2s7Cg== ------_=_NextPart_001_01C7F582.C3A05A33--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?85D4F2C294E8434CA0AF775741532686623694>