Date: Tue, 24 Jul 2007 10:10:12 -0700 From: "Jack Vogel" <jfvogel@gmail.com> To: freebsd-net@freebsd.org, paketix@bluewin.ch Subject: Re: em(4)/82571eb: fifo not dma'ed to host memory Message-ID: <2a41acea0707241010o52972359g94d8b336700664ab@mail.gmail.com> In-Reply-To: <2a41acea0707240925q6a4faef5r893cec8ad13ca7d9@mail.gmail.com> References: <20070717110156.GA37722@hbnnbsddev03.sharedtcs.net> <2a41acea0707170917n56e35cfbyc6e0bca094bfa729@mail.gmail.com> <20070718063801.GA40653@hbnnbsddev03.sharedtcs.net> <20070724114626.GA61209@hbnnbsddev03.sharedtcs.net> <2a41acea0707240925q6a4faef5r893cec8ad13ca7d9@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
WOW, I want to really thank you for finding this Patrick.
In Febuary 2006 Gleb introduced a drastic rename of our
adapter struct, it introduced a HUGE amount of diffs, we (Intel)
did not want that change and eventually it was backed out,
however it was in that delta that the bang got lost in this
logic, and all versions have picked up that error since.
There will be changes coming for both 6 and 7.
Jack
On 7/24/07, Jack Vogel <jfvogel@gmail.com> wrote:
> Hmmm, this looks like a bug, I will look more closely today.
> Sorry I have not been able to be more active with your problem,
> but things have been hectic with product launches and my own
> internal bug fixing. I will try to get this turned around quickly
> Patrick.
>
> Jack
>
>
> On 7/24/07, Patrick Oeschger <paketix@bluewin.ch> wrote:
> > OK - found the problem
> > the driver in 6.2-RELEASE has different code in em_identify_hardware
> > my hardware needs the memory access and bus master bits getting set by
> > the driver
> > beginning with 6.2-RELEASE this gets only set if PCIM_CMD_BUSMASTEREN is
> > *not set* and PCIM_CMD_MEMEN is *set* - it seems the 82571EB has both
> > parameters *not set* after init
> > is this a bug? feature? typo?
> >
> > --- /root/em/if_em.c.61 Mon Jul 23 12:17:35 2007
> > +++ if_em.c.62 Tue Jul 24 13:04:11 2007
> > static void
> > -em_identify_hardware(struct adapter * adapter)
> > +em_identify_hardware(struct adapter *adapter)
> > {
> > device_t dev = adapter->dev;
> >
> > /* Make sure our PCI config space has the necessary stuff set */
> > adapter->hw.pci_cmd_word = pci_read_config(dev, PCIR_COMMAND, 2);
> > - if (!((adapter->hw.pci_cmd_word & PCIM_CMD_BUSMASTEREN) &&
> > - (adapter->hw.pci_cmd_word & PCIM_CMD_MEMEN))) {
> > - printf("em%d: Memory Access and/or Bus Master bits were not set!
> > \n",
> > - adapter->unit);
> > + if ((adapter->hw.pci_cmd_word & PCIM_CMD_BUSMASTEREN) == 0 &&
> > + (adapter->hw.pci_cmd_word & PCIM_CMD_MEMEN)) {
> > + device_printf(dev, "Memory Access and/or Bus Master bits "
> > + "were not set!\n");
> > adapter->hw.pci_cmd_word |=
> > (PCIM_CMD_BUSMASTEREN | PCIM_CMD_MEMEN);
> > - pci_write_config(dev, PCIR_COMMAND, adapter->hw.pci_cmd_word, 2)
> > ;
> > + pci_write_config(dev, PCIR_COMMAND,
> > + adapter->hw.pci_cmd_word, 2);
> > }
> >
> > > thanks for your feedback jack - i tried the MSI stuff as first task this
> > > morning
> > > the interface init'ed without a problem and did not complain about MSI
> > > what i like most is that no IRQ is assigned now, everything seems to get
> > > handled in a TASKQ (IRQs were quite weird sometimes when having a bunch of
> > > interfaces...)
> > >
> > > but now the bad news: even with MSI it is not possible to recv any frames
> > > what makes me curious is, that the head pointer for receive fifo is *not*
> > > counting up after receiving a frame (good_packets counter incrementing)
> > > as far as i understand the 82571EB chipset manual, recv head counter inc
> > > is done in hardware after receiving a good_packet and having dma'ed it to
> > > host memory
> > > from if_em.c/em_process_receive_interrupts:
> > > * This routine executes in interrupt context. It replenishes
> > > * the mbufs in the descriptor and sends data which has been
> > > * dma'ed into host memory to upper layer.
> > >
> > > i did a quick test to check this on 6.1-RELENG by inserting a return
> > > at the top of em_process_receive_interrupts (see diff)
> > > register RDH0 is still incrementing for each good_packet received - even
> > > without receive interrupts being processed at the driver level
> > >
> > > freebsd61# sysctl dev.em.0.debug_info=1
> > > dev.em.0.debug_iemnfo: 0:-1 Adapter hardware address = 0xc4b7c928
> > > em0: CTRL = 0x81c0241 RCTL = 0x8002
> > > em0: RDBAH0 = 0x0 RDBAL0 = 0x3e0ff000
> > > em0: RDLEN0 = 0x1000 RDH0 = 0x3
> > > em0: RDT0 = 0xff RXDCTL = 0x10000
> > >
> > > ...so IMHO this increment (and also dma transfer to host memory) is done
> > > in hardware - RDH0 is not counting up when running under 7.0-CURRENT...
> > >
> > > erroneous init of chipset?
> > > anyone else running 7.0-CURRENT on a 82571EB pci express chipset?
> > >
> > > TIA for any help on this
> > > pat
> > >
> > > details for dmesg/if_stats/diff:
> > >
> > > pcib9: <PCI-PCI bridge> irq 19 at device 11.0 on pci2
> > > pcib9: secondary bus 9
> > > pcib9: subordinate bus 9
> > > pcib9: I/O decode 0x6000-0x6fff
> > > pcib9: memory decode 0xfd700000-0xfd7fffff
> > > pcib9: prefetched decode 0xfce00000-0xfcefffff
> > > pci9: <PCI bus> on pcib9
> > > pci9: physical bus=9
> > > found-> vendor=0x8086, dev=0x105e, revid=0x06
> > > bus=9, slot=0, func=0
> > > class=02-00-00, hdrtype=0x00, mfdev=1
> > > cmdreg=0x0000, statreg=0x0010, cachelnsz=16 (dwords)
> > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> > > intpin=a, irq=255
> > > powerspec 2 supports D0 D3 current D0
> > > MSI supports 1 message, 64 bit
> > > map[10]: type Memory, range 32, base 0, size 17, memory disabled
> > > map[14]: type Memory, range 32, base 0, size 17, memory disabled
> > > map[18]: type I/O Port, range 32, base 0, size 5, port disabled
> > > found-> vendor=0x8086, dev=0x105e, revid=0x06
> > > bus=9, slot=0, func=1
> > > class=02-00-00, hdrtype=0x00, mfdev=1
> > > cmdreg=0x0000, statreg=0x0010, cachelnsz=16 (dwords)
> > > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> > > intpin=b, irq=255
> > > powerspec 2 supports D0 D3 current D0
> > > MSI supports 1 message, 64 bit
> > > map[10]: type Memory, range 32, base 0, size 17, memory disabled
> > > map[14]: type Memory, range 32, base 0, size 17, memory disabled
> > > map[18]: type I/O Port, range 32, base 0, size 5, port disabled
> > > em0: <Intel(R) PRO/1000 Network Connection Version - 6.5.3> at device
> > > 0.0 on pci9
> > > pcib9: em0 requested memory range 0xfd700000-0xfd7fffff: good
> > > pcib2: em0 requested memory range 0xfd700000-0xfd7fffff: good
> > > pcib1: em0 requested memory range 0xfd700000-0xfd7fffff: good
> > > em0: Lazy allocation of 0x20000 bytes rid 0x10 type 3 at 0xfd700000
> > > em0: attempting to allocate 1 MSI vectors (1 supported)
> > > msi: routing MSI IRQ 256 to vector 49
> > > em0: using IRQ 256 for MSI
> > > em0: bpf attached
> > > em0: Ethernet address: 00:10:f3:0c:5b:2a
> > > em0: [FILTER]
> > > pci9: <network, ethernet> at device 0.1 (no driver attached)
> > >
> > > freebsd61# diff -u sys/dev/em/if_em.c.orig sys/dev/em/if_em.c
> > > --- sys/dev/em/if_em.c.orig Wed Jul 18 07:32:53 2007
> > > +++ sys/dev/em/if_em.c Wed Jul 18 07:48:42 2007
> > > @@ -277,6 +277,11 @@
> > >
> > > INIT_DEBUGOUT("em_probe: begin");
> > >
> > > + // prevent init of all em(4) devices except pci9/0/0
> > > + if(pci_get_bus(dev) != 0x09 || pci_get_slot(dev) != 0x00 ||
> > > + pci_get_function(dev) != 0x00)
> > > + return (ENXIO);
> > > +
> > > pci_vendor_id = pci_get_vendor(dev);
> > > if (pci_vendor_id != EM_VENDOR_ID)
> > > return(ENXIO);
> > > @@ -2906,6 +2911,8 @@
> > > u_int8_t eop = 0;
> > > u_int16_t len, desc_len, prev_len_adj;
> > > int i;
> > > +
> > > + return;
> > >
> > > /* Pointer to the receive descriptor being examined. */
> > > struct em_rx_desc *current_desc;
> > >
> > > On Tue, Jul 17, 2007 at 09:17:36AM -0700, Jack Vogel wrote:
> > > > As an experiment, search in if_em.c for 'msix' look at the logic there.
> > > > If the adapter is 82575 it will use msix, but the else clause has it
> > > > use msi ONLY if its '>82571', change that to >=82571 and let's
> > > > see if maybe using MSI will help you with your problems as I'm
> > > > pretty confident its interrupt related.
> > > >
> > > > I had intended on making this change shortly anyway, my tests
> > > > have shown the 571 to work fine this way.
> > > >
> > > > Let me know of the results.
> > > >
> > > > Jack
> > _______________________________________________
> > freebsd-net@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
> >
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2a41acea0707241010o52972359g94d8b336700664ab>