Date: Wed, 18 Dec 2013 02:13:23 -0000 From: "Steven Hartland" <killing@multiplay.co.uk> To: "John-Mark Gurney" <jmg@funkthat.com>, "Claude Marinier" <claude.marinier@cae.com> Cc: Michael Tuexen <Michael.Tuexen@lurchi.franken.de>, freebsd-net@freebsd.org Subject: Re: em3 no carrier Message-ID: <A5B6CADD759D4BDDA3DB5AB3719AF1E7@multiplay.co.uk> References: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> <D711A5E9-82F8-4495-9C5F-BF72BF0BD4D3@lurchi.franken.de> <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com> <20131218010342.GG99167@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message -----
From: "John-Mark Gurney" <jmg@funkthat.com>
To: "Claude Marinier" <claude.marinier@cae.com>
Cc: "Michael Tuexen" <Michael.Tuexen@lurchi.franken.de>; <freebsd-net@freebsd.org>
Sent: Wednesday, December 18, 2013 1:03 AM
Subject: Re: em3 no carrier
Claude Marinier wrote this message on Tue, Dec 17, 2013 at 19:43 -0500:
> -----Original Message-----
> From: Michael Tuexen [mailto:Michael.Tuexen@lurchi.franken.de]
> Sent: Tuesday, December 17, 2013 5:24 PM
> To: Claude Marinier
> Cc: freebsd-net@freebsd.org
> Subject: Re: em3 no carrier
>
> On Dec 17, 2013, at 10:51 PM, Claude Marinier <claude.marinier@cae.com> wrote:
>
> >
> >>> -----Original Message-----
> >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net]
> >>> Sent: Tuesday, December 17, 2013 11:40 AM
> >>> To: Claude Marinier
> >>> Cc: freebsd-net at freebsd.org
> >>> Subject: Re: em3 no carrier
> >>>
> >>> Hi,
> >>>
> >>> Can you give us more informations about the network card / chips ?
> >>> Did you found any error in /var/log/ ?
> >>>
> >>> - rodrigo
> >>
> >> My apologies, I missed that bit of information.
> >>
> >> Yes, there is an error in /var/log/messages.
> >>
> >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer
> >> space available Dec 17 11:33:47 WANemu last message repeated 4 times
> >> Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57
> >> WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last
> >> message repeated 59 times
> >>
> >>
> >> Also note that the HP NC364T uses an Intel 82571EB chipset.
> >>
> >>> On 17/12/13 11:27 -0500, Claude Marinier wrote:
> >>>> Hi,
> >>>>
> >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2
> >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port
> >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports are
> >>>> connected to routers in a lab.
> >>>>
> >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep
> >>>> 26 22:50:31 UTC 2013 root at
> >>>> bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> >>>>
> >>>> Dec 16 17:13:19 WANemu kernel: em3: <Intel(R) PRO/1000 Network
> >>>> Connection 7.3.8> port 0x6020-0x603f mem
> >>>> 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on
> >>>> pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt
> >>>> Dec 16 17:13:19 WANemu kernel: em3: Ethernet address:
> >>>> e8:39:35:13:21:6e
> >>>>
> >>>> The four em interfaces are configured the same way (lines in
> >>>> rc.conf are identical except for IP address.
> >>>>
> >>>> ifconfig_em3="inet x.y.113.197/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em0="inet x.y.113.14/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em2="inet x.y.113.109/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em1="inet x.y.113.189/29 media 100baseTX mediaopt full-duplex"
> >>>>
> >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows
> >>>> "no carrier" for em3. Oddly, it also shows autoselect.
> >>>>
> >>>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6d
> >>>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15
> >>>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex>
> >>>> status: active
> >>>> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6c
> >>>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191
> >>>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex>
> >>>> status: active
> >>>> em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6f
> >>>> inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111
> >>>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex>
> >>>> status: active
> >>>> em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6e
> >>>> inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199
> >>>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex> (autoselect)
> >>>> status: no carrier
> >>>>
> >>>> I have tried three Ethernet cables with no difference. This
> >>>> morning, I briefly booted Puppy Linux from USB and manually
> >>>> configured the four Ethernet interfaces. They all work (ping good
> >>>> to all four routers); the Cisco on em3 shows happy lights. I then
> >>>> booted back into FreeBSD, the problem with em3 remains. I have not
> >>>> had any success searching with Google; I may not have used suitable search terms.
> >>>>
> >>>> One more odd symptom: bmon has trouble with these interfaces.
> >>>>
> >>>> em0è95^S!m on WANemu
> >>>> Name
> >>>> qqqqqqqqqqqqqqqqqqqqqqqq
> >>>> WANemu (local)
> >>>> 0 em0è95^S!m
> >>>> 1 em1è95^S!l
> >>>> 2 em2è95^S!o
> >>>> 3 em3è95^S!n
> >>>> 4 usbus0
> >>>> 5 usbus1
> >>>> 6 usbus2
> >>>> 7 usbus3
> >>>> 8 usbus4
> >>>> 9 usbus5
> >>>> 10 ipfw0
> >>>> 11 lo0
> >>>>
> >>>> It looks like FreeBSD is somehow mishandling these interfaces.
> >>>> Before purchasing it, I searched with Google and found reports of
> >>>> people using this device without problems. I do not know how to proceed.
> >>>>
> >>>> Thank you for your help.
> >
> > Oddly, replacing em3 with bce1 left the system with only two
> > functionning interfaces. The order of the interface definitions in
> > rc.local makes a difference. The worst order is bce1, em0, em2, em1
> > which left only
> > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows
> > the three 'em' interfaces to work. In both, bce1 is not working (no
> > carrier). Changing the order to em0, em1, em2, bce1 did not improve
> > the situation.
Silly question but your not running out of mbufs on init are you?
I've seen a very similar issues before and increasing mbufs and in
some cases jumbos fixed the issue.
I would also strongly avoid using hard coded speed / duplex and if you
do need to use it ensure both ends are hard coded otherwise it just
wont work.
Regards
Steve
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A5B6CADD759D4BDDA3DB5AB3719AF1E7>