Date: Fri, 31 Oct 2014 13:19:01 +0100 From: Julien Charbon <jch@freebsd.org> To: freebsd-net <freebsd-net@freebsd.org> Cc: Oleg Bulyzhin <oleg@freebsd.org>, Ryan Stone <rysto32@gmail.com>, "K. Macy" <kmacy@freebsd.org>, "De La Gueronniere, Marc" <mdelagueronniere@verisign.com> Subject: Re: buf_ring in HEAD is racy Message-ID: <54537E35.2060108@freebsd.org> In-Reply-To: <CAHM0Q_PCr=8e26E%2BsTM_9GSZe0uOQyEw_UeHyhpbF-J81-AXyw@mail.gmail.com> References: <CAFMmRNyJpvZ0AewWr62w16=qKer%2BFNXUJJy0Qc=EBqMnUV3OyQ@mail.gmail.com> <20131226175410.GA15332@lath.rinet.ru> <D06D9912.1507F%mdelagueronniere@verisign.com> <CAHM0Q_PCr=8e26E%2BsTM_9GSZe0uOQyEw_UeHyhpbF-J81-AXyw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ue48W3K8oNOB8hpL4MxaQn4mP7sKkFQfn Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, On 30/10/14 20:39, K. Macy wrote: >> I also suspect there are further problems with buf_ring. A full wrap >> around of the atomically swapped value is possible. I.e. the code thin= ks >> it just atomically updated a head/tail index when in fact a full wrap >> around occurred leading to undefined land. A relatively simple way to >> avoid this is to only mask on ring array access, and to let the >> head/tail/prod/cons indices overflow the array. >=20 > Up until Rui Paulo complained to me of packet drops with buf_ring a > couple of days ago I had thought that this patch had been committed. > This patch (now 273866) fixes the problem for him. Without further > scrutiny and testing I won't provide the UL guarantee for > buf_ring_enqueue, but this is a clear improvement. I have tested r273866 fix against our traffic and it seems it fixes the ENOBUFS issue for us too. Below the Dtrace script used for checking who was returning ENOBUFS: $ cat enobufs.d fbt::sendit:return /arg1 =3D=3D ENOBUFS/ { @senditENOBUFS[execname,pid] =3D count(); } fbt::ixgbe_mq_start:return /arg1 =3D=3D ENOBUFS/ { @ixgbeStartENOBUFS[execname,pid] =3D count(); } fbt::ip_output:return /arg1 !=3D 0/ { @ipOutputErr[execname,pid] =3D count(); } END { printf("\nsendit ENOBUFS:\n"); printa(@senditENOBUFS); printf("\nixgbe_mq_start ENOBUFS:\n"); printa(@ixgbeStartENOBUFS); printf("\nip_output Errors:\n"); printa(@ipOutputErr); } The result without the fix: $ dtrace -s enobufs.d dtrace: script 'enobufs.d' matched 4 probes ^C CPU ID FUNCTION:NAME 0 2 :END sendit ENOBUFS: server 1779 47 ixgbe_mq_start ENOBUFS: server 1779 47 ip_output Errors: server 1779 47 Thus 47 ENOBUFS errors, all returned by ixgbe_mq_start() which calls drbr_enqueue(). And result with the fix applied: $ dtrace -s enobufs.d dtrace: script 'enobufs.d' matched 4 probes ^C CPU ID FUNCTION:NAME 3 2 :END sendit ENOBUFS: ixgbe_mq_start ENOBUFS: ip_output Errors: Thus 0 ENOBUFS errors. I will let Marc review this fix further to check his full wrap case. Thanks. -- Julien --ue48W3K8oNOB8hpL4MxaQn4mP7sKkFQfn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUU348AAoJEKVlQ5Je6dhxYGwIALJihOjnir20oD5/J8sDK7Vc 7ypJEga3zc296bM5v5F4r3wRdRH/3iEmVI1f+JI+IU4o40JqhwDNsMBz2DZ41oTO iTtK7KD1v9BxOq8YD/WfMKZmC6Y0yGjd8LRYXZ8YiZLVnbhHioKMvkqn3LMBGOfE 68cJqPgYqEGSURk/lynHAaspS6l1qX0bUjBXDO3Gv79YmO7AcQuFEsc3+tkhfTPE 649ecEgNPDh3uVKT9D9hUdxslw5Rhe97KAj35cEbyQoczMZCvEpoL3VIOhaIYifN djf5J4VsJdcanE3vWkDm1MFxDEsVRJ+CntsMZ7PXSscGjgbgO4pu2Bc3m5UQ9EE= =bm38 -----END PGP SIGNATURE----- --ue48W3K8oNOB8hpL4MxaQn4mP7sKkFQfn--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54537E35.2060108>