From owner-freebsd-current@FreeBSD.ORG Tue Mar 13 12:02:46 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0CBB216A402 for ; Tue, 13 Mar 2007 12:02:46 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.freebsd.org (Postfix) with ESMTP id C123A13C48A for ; Tue, 13 Mar 2007 12:02:45 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1HR5iG-0000YS-1p; Tue, 13 Mar 2007 14:02:44 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: pyunyh@gmail.com In-reply-to: Your message of Tue, 13 Mar 2007 20:37:41 +0900 . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Mar 2007 14:02:43 +0200 From: Danny Braniss Message-ID: Cc: freebsd-current@freebsd.org Subject: Re: nfe/PXE problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Mar 2007 12:02:46 -0000 > On Tue, Mar 13, 2007 at 09:10:38AM +0200, Danny Braniss wrote: > > > > > well, I can PXE boot this box if I use an fxp NIC, with the nfe I tracked the > > > problem > > > to sys/nfsclient/nfs_diskless, where in nfs_setup_diskless(void) > > > it does not find the nfe interface, ie, something does not match, but the > > > interface > > > was detected!. > > > i'll try and do some more debugging. > > > > > ok, I found the problem, in nfs_diskless.c nfs_setup_diskless(), > > there is a loop to search for the interface that was used to boot from, > > and no match is found because the hadrware ethernet address > > in the nfe is in the wrong byte order! and so the bcmp(...) fails. > > Good catch! > > > Interestingly, if booting NOT via PXE the Ethernet address is OK! > > > > nfe0: port 0xdc00-0xdc07 mem > > 0xfe02c000-0xfe02cfff irq 22 at device 20.0 on pci0 > > nfe0: Ethernet address: 00:18:f3:a9:6c:57 > > and via PXE: > > nfe0: Ethernet address: 57:6c:a9:f3:18:00 > > > > can someone with the right knowledge fix this? > > > > AFAIK there is no known way to get an ethernet address via EEPROM on > NVIDIA NIC so nfe(4) reads specific address registers to get ethernet > address as a workaround. The address registers are filled > automatically by hardware after hardware reset. This type of > acquisition of ethernet address is *NOT* correct way as it could get > a fake etherent address since adiministor can program other ethernet > address into the NIC.(e.g. ifconfig(8) can change ethernet address.) > Therefore nfe(4) is very careful to save original ethernet address > in device attach phase and restores the ethernet address at device > detach time. > To make matters worse, it seems that ethernet address in NVIDIA NIC > is loaded backwards into registers so nfe(4) corrects it in > nfe_init_locked(). However, it seems PXE code does not do necessary > swapping for ethernet address and does not restore original MAC > address in the end of PXE phase so the NIC will have bogusly > programmed MAC address. When kernel boots and nfe(4) is attached it > will load the bogus ethernet address which was already programmed by > PXE. If /etc/start_if.nfe0 is honored in diskless boot you can override > the ethernet address to use the same ethernet address as PXE did. > > Even if there is a way to get an ethernet address from EEPROM, PXE > should be fixed first since it uses bogus ethernet address in booting > stage. If I encounter the same situation I would contact vendor for > updated PXE image for the NVIDIA NIC. If that is not available I think > the only remaining option would be adding a new tunable to swap the > ethernet address. > > I'm not familiar with PXE and I din't use PXE for a long time so I > could be completely wrong. not completely :-) the DHCP/PXE boot works with the 'correct ethernet address'. this is because I use the static/registered mac, and so a bogus mac address can't get past the DHCP/PXE stage. so, the PXE is using the correct mac address. I can't use /etc/... because of the horse-cart problem, it's booting diskless, and the 'disk' is nfs readable via the NIC, but the NIC is not working the question now, is to see if the nfe driver can check if it was used by the PXE, and flip the address. or have NVIDIA comeup with a patch ... thanks, danny