From owner-freebsd-hackers  Sun Sep 14 17:30:23 1997
Return-Path: <owner-freebsd-hackers>
Received: (from root@localhost)
          by hub.freebsd.org (8.8.7/8.8.7) id RAA14213
          for hackers-outgoing; Sun, 14 Sep 1997 17:30:23 -0700 (PDT)
Received: from ocean.campus.luth.se (ocean.campus.luth.se [130.240.194.116])
          by hub.freebsd.org (8.8.7/8.8.7) with ESMTP id RAA14202
          for <hackers@FreeBSD.ORG>; Sun, 14 Sep 1997 17:30:12 -0700 (PDT)
Received: (from karpen@localhost)
	by ocean.campus.luth.se (8.8.5/8.8.5) id CAA01290;
	Mon, 15 Sep 1997 02:33:48 +0200 (CEST)
From: Mikael Karpberg <karpen@ocean.campus.luth.se>
Message-Id: <199709150033.CAA01290@ocean.campus.luth.se>
Subject: Re: Memory leak in getservbyXXX?
In-Reply-To: <199709122115.OAA25502@usr08.primenet.com> from Terry Lambert at "Sep 12, 97 09:15:34 pm"
To: tlambert@primenet.com (Terry Lambert)
Date: Mon, 15 Sep 1997 02:33:48 +0200 (CEST)
Cc: julian@whistle.com, mike@smith.net.au, gram@cdsec.com, hackers@FreeBSD.ORG
X-Mailer: ELM [version 2.4ME+ PL31H (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: owner-freebsd-hackers@FreeBSD.ORG
X-Loop: FreeBSD.org
Precedence: bulk

According to Terry Lambert:
> > > > It does seem like a memory leak, as the memory use reported by top grows
> > > > over time. I have memory allocation debugging code which confirms that
> > > > I have no leaks in my code (at least of C++ objects), and the fact that
> > > > older sites have been running for months seems to confirm this.
> > > 
> > > OK.  More to the point then it sounds like it's a memory leak due to
> > > some change in stdio.  That's getting slightly easier to chase I 
> > 
> > they aren't using the threaded libc are they?
> 
> Exactly my thought as well...
> 
> To the user: you should note that the threaded libc differs from the
> standard libc in that, instead of returning pointers to function
> modified static data areas, most functions which formerly did just
> that now return allocated buffers which contain the results, and it
> is now the user's responsibility to free these buffers.
> 
> The use of static data buffers in libc is a long-standing abomination,
> blessed by ANSI and codified in stone by POSIX.  Assume any man page
> which states a pointer to a static data area is returned is actually
> allocating a data area and expecting the caller to free it, whenever
> you are using a thread-safe library.

That seems... DUMB! Returning something that the user must free instead of
static buffers, when that's the normal behaviour is just very very bad.

Three words: Thread local storage.
Let each tread have its OWN static buffer. This is actually nicer in some
ways then having the programmer pass buffers to pointers, etc, in *_r
versions of calls. It's nicer because it's less work for the user of the
library. And returning something the user must free is just out of the
question. That's ugly beyond belief. Everyone deals with their own memory.
Libraries too.

Ok, now you say "But what if we have real threads which have a stack and
TLS on the same memory addresses! Then you can't share returned information
between threads!". Yes, I say.

Something like this: (Not thought about too long. Might contain errors.)

All that is needed is that the library uses (for efficiency) to use a struct
with all the libraries static buffers collected in it as entries. Then just
allocate that struct with malloc, and make a pointer in TLS point to it.
Like:

struct z {
  ...
  char addr2ascii_buffer[64];
  ...
};

inline struct z *getstaticdatapointer(void) {
  /* don't remember these TLS call exactly... */
  struct z *ptr = pthread_getkey(LIBC_STATIC_KEY);
  if (ptr == NULL) {   /* or allocate always at pthread_create */
     ptr == malloc( sizeof(struct z) );
     pthread_setkey(ptr, LIBC_STATIC_KEY);
  }
  return ptr;
}

Accessed like:

  char *buffer_to_return = getstaticdatapointer().addr2ascii_buffer;


Ok... exact implementaion is not important. But that way you make it as
simple as possible for the programmer (he doesn't have to care about if
he's MT or not, and there is no messing with allocating/freeing memory).
And I see no problems with it. Speed/space possibly. But I think both are
reasonable.

Comments?

  /Mikael -- donning asbesto suit