Date: Mon, 9 Nov 2020 15:15:29 -0800 From: John-Mark Gurney <jmg@funkthat.com> To: "Alexander V. Chernikov" <melifaro@ipfw.ru> Cc: freebsd-arch <freebsd-arch@freebsd.org> Subject: Re: Versioning support for kernel<>userland sysctl interface Message-ID: <20201109231529.GH31099@funkthat.com> In-Reply-To: <428251604959994@mail.yandex.ru> References: <356181604233241@mail.yandex.ru> <20201102221330.GS31099@funkthat.com> <428251604959994@mail.yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Alexander V. Chernikov wrote this message on Mon, Nov 09, 2020 at 22:28 +0000:
> 02.11.2020, 22:13, "John-Mark Gurney" <jmg@funkthat.com>:
> > Alexander V. Chernikov wrote this message on Sun, Nov 01, 2020 at 12:47 +0000:
> >> I would like to propose a change [1] that introduces versioning support for the data structures exposed to userland by sysctl interface.
> >>
> >> We have dozens of interfaces exposing various statistics and control data by filling in and exporting structures.
> >> net.inet6.icmp6.stats or net.inet6.icmp6.nd6_prlist can be a good examples of such interaction.
> >
> > We also need to decide the policy on dealing w/ support for these
> > data structures going forward... Because if we do the simple, default
> > policy of all userland apps can handle all structures, and kernel can
> > produce all structures, we now have an unbounded growth of complexity
> > and testing...
> I totally agree. While backward compatibility is important, it should not impose notable technical debt. I had the following as my mental model:
> * the code should be organised to support output for the latest version.
> * There should be a separate, isolatable, piece of code that converts from latest to n-1 (which can be chained: from n-1 to n-2 and so on)
> * when introducing changes we should garden older versions by COMPAT_X defines.
Yeah, if we restrict the code to COMPAT_x for the existing versions,
and ensure that it doesn't change, it isn't TOO terrible, but still,
the likelyhood of people writing tests and verifying that they work
to make sure that the compat code works for all n-x versions isn't
great... it's doable, but I dobut most people are going to put in the
effort..
> > I do understand the desire to solve this problem, but IMO, this solution
> > is too simple, and dangerous to unbounded growth above.
> >
> > While I do like it's simplicity, one idea that I've had, while being a
> > bit more complex, has the ability to handle modification in a more
> > compatible way.
> >
> > Since we have dtrace, one of the outputs of dtrace is ctf, which allows
> > use to convey the type and structure information in a machine parseable
> > format. The idea is that each sysctl oid (that supports this) would
> > have the ability to fetch the ctf data for that oid. The userland would
> > then be able to convert the members to the local members of a similar
> > struct. A set of defaults could also be provided, allowing new fields
> > to have sane initial values.
> >
> > As long as the name of a structure member is never reused for a different
> > meaning, this will get us most of the way there, in a much cleaner
> > method...
> >
> > I do realize that this isn't the easiest thing, but the tools to do this
> > are in the tree, and would solve this problem, IMO, in a way that is a
> > lot more maintainable, and long term than the current proposal.
> >
> > Other solution, use ctf data to produce nvlist generation/consumption
> > code for a structure... The data transfered would be larger, but also
> > more compatible...
> I do like idea on the self-documenting approach. It addresses append-only case nicely, but that's not always the case.
> For example, in the initially-discussed icmp6 stats we have 256 64-bit counters representing icmp6 protocol historgram, resulting in 4k frame being allocated on stack for the current kernel implementation. If in the future our icmp6 kernel implementation changes and we won't be able to provide this counters, eventually we would want to remove all these counters from the structure. I'm not sure how can this be addressed without some sort of versioning scheme.
So the bit that gets the ctf data would also have an nvlist (or
something) that contains the defaults for when fields are removed...
So, initally:
struct foo {
int x;
int y;
};
Then it gets changed to:
struct foo {
int x;
int y;
int z;
};
This is easy, the z will be included, and transmitted, but be ignored
by older code, then when it changes to:
struct {
int y;
int z;
};
The ctf data would be something like:
<ctf data>,<nvlist defaults>
Where nvlist defaults is:
x: -1
So, the consuming code would set the defaults from the nvlist first,
then set the fetch data, so that x gets set to some default value.
With a few simple rules like this, handling deletions is not a problem
when the older code is expecting it. If a variable must always have
a value that "must" be correct (and a default cannot be set), then
another member needs to be added (I think ctf handles bit fields
properly) that says if that member is valid... when it gets removed,
that is valid flag gets set to zero, and then the old code knows not
to handle it.
> > Overall, using bare structures is an ABI compatibility nightmare that
> > should be fixed in a better method.
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20201109231529.GH31099>