Date: Fri, 15 Nov 1996 14:01:45 -0800 (PST) From: owner-hackers-digest To: freebsd-hackers-digest@FreeBSD.ORG Subject: hackers-digest V1 #1645 Message-ID: <199611152201.OAA20062@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
hackers-digest Friday, 15 November 1996 Volume 01 : Number 1645
In this issue:
Re: Sockets question...
2.1.5-stable build fails in gdb, mrouted, named, nslookup
Re: Sockets question...
Re: Sockets question...
Re: Sockets question...
Re: Sockets question...
earlier "holographic shell" in 2.2-ALPHA install procedure
Re: Sockets question...
Re: Sockets question...
Kernel Commands
Re: Sockets question...
Re: FreeBSD 2.2-ALPHA is now available.
Re: Sockets question...
Re: Sockets question...
Re: Sockets question...
Re: Sockets question...
Re: Sockets question...
----------------------------------------------------------------------
From: Joe Greco <jgreco@brasil.moneng.mei.com>
Date: Fri, 15 Nov 1996 14:21:51 -0600 (CST)
Subject: Re: Sockets question...
> >> Did you receive a signal? That is known to cause similar behaviour on
> >> SunOS...
> >
> >Can't. Any signal on that process is a SERIOUS error; its a DBMS! We have
> >a generic "oh shit" trap for all signals set; it does not go off.
>
> Don't do that for SIGIO. Anyone can send SIGIO to any process.
Bruce,
Could you clarify what you mean? Obviously "anyone" can not do that...
Sorry, I am just missing your meaning.
... JG
------------------------------
From: Chris Shenton <cshenton@it.hq.nasa.gov>
Date: Fri, 15 Nov 1996 15:37:02 -0500
Subject: 2.1.5-stable build fails in gdb, mrouted, named, nslookup
Flame me (informatively) if I'm doing this wrong, but...
I'm running 2.1.5-RELEASE and recently supped -stable. I do a "make
world" from /usr/src, and get a few compilation errors that I probably
shouldn't. Nothing fatal, but gdb, mroute, named, and nslookup
*should* build.
Can anyone enlighten me? Thanks.
cd /usr/src/
make -k
[...]
===> gnu/usr.bin/gdb/gdb
cc -O -I/home/src/gnu/usr.bin/gdb/gdb/. -I/usr/include/readline -I/home/src/gnu/usr.bin/gdb/gdb/../bfd -o gdb annotate.o blockframe.o breakpoint.o buildsym.o c-lang.o c-typeprint.o c-valprint.o ch-lang.o ch-typeprint.o ch-valprint.o coffread.o command.o complaints.o copying.o core.o coredep.o corelow.o cp-valprint.o dcache.o dbxread.o demangle.o disassemble.o dis-buf.o dwarfread.o elfread.o environ.o eval.o exec.o expprint.o findvar.o fork-child.o freebsd-nat.o gdbtypes.o i386-dis.o i386-pinsn.o i386-tdep.o infcmd.o inflow.o infptrace.o infrun.o inftarg.o init.o kcorelow.o language.o m2-lang.o m2-typeprint.o m2-valprint.o main.o maint.o mem-break.o minsyms.o objfiles.o parse.o printcmd.o remote.o remote-utils.o solib.o source.o stabsread.o stack.o symfile.o symmisc.o symtab.o target.o thread.o top.o typeprint.o utils.o valarith.o valops.o valprint.o values.o version.o serial.o ser-unix.o mdebugread.o c-exp.tab.o ch-exp.tab.o m2-exp.tab.o compat_que.o -lreadline -ltermca!
p !
- -lgnuregex -L/home/src/gnu/usr.bin/gdb/gdb/../libiberty/obj -liberty -L/home/src/gnu/usr.bin/gdb/gdb/../bfd/obj -lbfd -L/home/src/gnu/usr.bin/gdb/gdb/../mmalloc/obj -lmmalloc
utils.o: Undefined symbol `_vasprintf' referenced from text segment
utils.o: Undefined symbol `_vasprintf' referenced from text segment
*** Error code 1 (continuing)
`all' not remade because of errors.
===> usr.sbin/mrouted/mrouted
cc -O -I/home/src/usr.sbin/mrouted/mrouted/.. -DRSRR -DRSRR -o mrouted config.o cfparse.o main.o route.o vif.o prune.o callout.o rsrr.o -L/home/src/usr.sbin/mrouted/mrouted/obj/../common -lmrouted
main.o: Undefined symbol `_configfilename' referenced from text segment
vif.o: Undefined symbol `_configfilename' referenced from text segment
vif.o: Undefined symbol `_config_vifs_from_file' referenced from text segment
*** Error code 1 (continuing)
`all' not remade because of errors.
===> usr.sbin/named
cc -O -o named version.o db_dump.o db_load.o db_lookup.o db_reload.o db_save.o db_update.o db_secure.o db_glue.o ns_forw.o ns_init.o ns_main.o ns_maint.o ns_req.o ns_resp.o ns_sort.o ns_stats.o ns_validate.o ns_ncache.o storage.o dmalloc.o tree.o -lutil
ns_main.o: Undefined symbol `_Version' referenced from text segment
ns_main.o: Undefined symbol `_Version' referenced from text segment
*** Error code 1 (continuing)
`all' not remade because of errors.
===> usr.sbin/nslookup
cc -O -o nslookup main.o commands.o getinfo.o debug.o send.o skip.o list.o subr.o -ll
main.o: Undefined symbol `_yylex' referenced from text segment
subr.o: Undefined symbol `_yyin' referenced from text segment
subr.o: Undefined symbol `_yyrestart' referenced from text segment
*** Error code 1 (continuing)
`all' not remade because of errors.
------------------------------
From: Joe Greco <jgreco@brasil.moneng.mei.com>
Date: Fri, 15 Nov 1996 14:37:11 -0600 (CST)
Subject: Re: Sockets question...
> In message <199611151706.KAA26239@phaeton.artisoft.com> Terry wrote:
> >How do I read into a structure on a machine that demands aligned
> >data access?
>
> You read into an intermediate buffer and copy it. You have to convert from
> network to machine representation anyway, so this isn't (much) more overhead.
> Or you use UDP if you want a record-oriented protocol.
Actually... I usually found it easier to do the following.
#include <stdio.h>
/*
* One dull variant of Joe's "xread" function.
*/
int xread(fd, buf, siz)
int fd;
char *buf;
int siz;
{
int rval;
int chrs = 0;
while (siz) {
if ((rval = read(fd, buf, siz)) < 0) {
return(rval);
}
chrs += rval;
siz -= rval;
buf += rval;
}
}
int main()
{
int rval;
struct big_ugly {
yadda;
yadda;
yadda;
} data;
if ((rval = xread(fd, (char *)&data, sizeof(data))) != sizeof(data)) {
fprintf(stderr, "Help me, I am on fire, xread returned %d\n",
rval);
}
}
This of course assumes you either do not need to do byte reordering, or do
it elsewhere.
... JG
------------------------------
From: Bill Fenner <fenner@parc.xerox.com>
Date: Fri, 15 Nov 1996 12:44:03 PST
Subject: Re: Sockets question...
>Actually... I usually found it easier to do the following.
And that's what Terry's complaining about; if you have alignment
requirements when writing big_ugly->yadda, those alignment requirements
might not be met by the read() inside of xread().
Bill
------------------------------
From: Joe Greco <jgreco@brasil.moneng.mei.com>
Date: Fri, 15 Nov 1996 14:59:34 -0600 (CST)
Subject: Re: Sockets question...
> >Actually... I usually found it easier to do the following.
>
> And that's what Terry's complaining about; if you have alignment
> requirements when writing big_ugly->yadda, those alignment requirements
> might not be met by the read() inside of xread().
Ummm... and the problem is...?
As far as I am aware, byte oriented data can be written to unaligned
addresses on any UNIX architecture that I have seen.
xread is explicitly called with what is clearly a byte oriented buffer.
If you are possibly worried about something such as the atomicity of
reads (potentially valid in a threaded environment, or one using shared
memory), I agree that there may be some concern. Since it is not clear
to _me_ that such atomicity of access would be valid under the same
circumstances even with read(), I would probably code around the
situation anyways.
Is there some other problem that I am missing? I've done this sort of
things for several years now...
... JG
------------------------------
From: Bruce Evans <bde@zeta.org.au>
Date: Sat, 16 Nov 1996 07:59:35 +1100
Subject: Re: Sockets question...
>> on system 1 with 32MB RAM: the kernel panics with a double fault
>
>This is, I think, expected behaviour for a page which is not present but
>for which a kmem mapping exists. This happens because the /kernel file
>is mapped to /dev/kmem.
Not in FreeBSD. /kernel is an ordinary, unmapped file.
>If you try this with a different file (one without an established mapping),
>what happens?
Same. It was a bug in i586_copyout.
>
>> on system 2 with 8MB RAM: read returns -1.
This worked because it's an i486. The memory size was irrelevant. This
shows why everyone should include full information information in bug
reports.
Bruce
------------------------------
From: Mike Haertel <haertel@incognito.intel.com>
Date: Fri, 15 Nov 1996 13:04:52 -0800 (PST)
Subject: earlier "holographic shell" in 2.2-ALPHA install procedure
I would like the ability to launch the "emergency
holographic shell" earlier in the install process,
say from a menu entry somewhere. Mainly I would
like to be able to launch it before "ifconfig"
is called in network installs.
------------------------------
From: Terry Lambert <terry@lambert.org>
Date: Fri, 15 Nov 1996 13:58:03 -0700 (MST)
Subject: Re: Sockets question...
> >So at the "read" interface, you *can* count on it arriving in the
> >same sized chunks as you wrote it
>
> No, you can *never* count on that, since non-blocking reads from a stream
> socket return as much data as is available, which could be less than you
> asked for. See soo_read() (or soo_rw() in earlier BSD's) and soreceive().
By default, the sockets are blocking. You have to go out of your
way to make them non-blocking (ie: loading the gun before you can shoot
yourself in the foot).
> 4.4BSD introduced the MSG_WAITALL flag, so if you use recv() or any of its
> friends you can ask for your whole request to be performed. This is, of
> course, not portable, and MSG_WAITALL won't even do the trick if your
> request is larger than the socket's high water mark (e.g. SO_RECVBUF).
This makes sense.
But on a blocking socket, it doesn't make sense to have to issue multiple
system calls to read chunks of a whole message when you aren't going to
do anything with it until all the reads have been satisfied?
It makes sense to do it on the server, if you have a server getting
messaged from a bunch of sources, but that's just moving the packet
assembly into user space. You'd expect the server to have a higher
premium on interleaving operations than on atomicity of transactions.
Even so, this only becomes a problem if you send "outrageously large"
packets in a single write.
You'd think that you could move it back down into kernel space by the
select not coming true until all frags had been reassembled. I could
see SO_RECVBUF putting a limit on your biggest message size in that
case.
The MSG_WAITALL seems like, even without the flag, it would be the
default behaviour for the read. The alternative is to push the
"frag" assembly ("frags" in this case being buffers larger than
SO_RECVBUF cut into SO_RECVBUF) up into the user program, an dictate
to the user how he must code a state machine in user space in order
to use the interface. Ugh, copy overhead. Bleah. 8-(.
I suppose one very real possibility is that he's trying to send huge
packets... most of the drivers don't support them, and the DEC driver
supports them only with a couple of patches that Matt and Amancio
beat out between them a while back...
If it's huge packets, he's probably SOL, and needs to redesign his
stream formatting.
Anyway, let's wait and see what clarifications he has to offer before
we speculate ourselves to death. 8-).
Terry Lambert
terry@lambert.org
- ---
Any opinions in this posting are my own and not those of my present
or previous employers.
------------------------------
From: Terry Lambert <terry@lambert.org>
Date: Fri, 15 Nov 1996 14:09:34 -0700 (MST)
Subject: Re: Sockets question...
> In message <199611151858.LAA26626@phaeton.artisoft.com> you write:
> >So at the "read" interface, you *can* count on it arriving in the
> >same sized chunks as you wrote it
>
> No, you can *never* count on that, since non-blocking reads from a stream
> socket return as much data as is available, which could be less than you
> asked for. See soo_read() (or soo_rw() in earlier BSD's) and soreceive().
>
> 4.4BSD introduced the MSG_WAITALL flag, so if you use recv() or any of its
> friends you can ask for your whole request to be performed. This is, of
> course, not portable, and MSG_WAITALL won't even do the trick if your
> request is larger than the socket's high water mark (e.g. SO_RECVBUF).
Maybe this is where I'm confused. I was under the impression that
the MSG_WAITALL was for messages that had to be passed up to free
up the receive buffer, and was put there to allow the copy to the
user buffer to occur in SO_RECVBUF-sized chunks.
Again, we aren't really interested in the non-blocking socket case...
Terry Lambert
terry@lambert.org
- ---
Any opinions in this posting are my own and not those of my present
or previous employers.
------------------------------
From: Tony Sterrett <tony@nlanr.net>
Date: Fri, 15 Nov 1996 13:23:20 -0800 (PST)
Subject: Kernel Commands
Hello everybody:
Does there exist systems calls used in the kernel to do things
like disabling the cache(Pentum), allocating memory etc.
Can someone do me the flavor of posting and call for disabling
cache and maybe provide a reference to this class of commands.
Thanks in Advance.
Tony
------------------------------
From: Terry Lambert <terry@lambert.org>
Date: Fri, 15 Nov 1996 14:11:24 -0700 (MST)
Subject: Re: Sockets question...
> I'm telling you that you have no credibility with your statements.
[ ... ]
> I never stated it shouldn't. My statement was that your 'blanket'
> statements in trying to describe his problem show that you like to see
> your email more than you want to fix the problem.
[ ... ]
> I never stated he didn't know what he was doing, simply that you have
> shown an obvious lack of understanding of the problem.
Well Nate, you stud, you...
If you can solve his problem in 3 notes... SOLVE HIS PROBLEM!
Otherwise you are doing exactly what you accuse me of doing.
Terry Lambert
terry@lambert.org
- ---
Any opinions in this posting are my own and not those of my present
or previous employers.
------------------------------
From: "Rodney W. Grimes" <rgrimes@GndRsh.aac.dev.com>
Date: Fri, 15 Nov 1996 13:26:22 -0800 (PST)
Subject: Re: FreeBSD 2.2-ALPHA is now available.
> > > 4 port does not, 6 port does.
> > >
> > > Unless the product offering has changed lately.
> >
> > Thanks Joe, combining your statement with what another person said
> > defanitly qualifies the fact that the BB1004 does not have modem
> > support, and that the IOAT66 (6-port) does.
>
> Actually cruising the Boca Web site is handy... ( :-) :-) )
>
> http://www.bocaresearch.com/docs/prodlist.htm
...
>
> Unfortunately their use of "RJ-45" is incorrect, it's actually a ten pin
> connector, allowing for full modem control.
It is correct, it is called an RJ-45/10, I know, I had to order a bunch
of them to make cables when Boca was backorder 30 days on them. You also
need a special set of die's for your crimp tool, but it is infact known
as an RJ-45, they just left the /10 off :-(.
> > Jordan, can you please list this in the serial board section of the
> > release/whatever notes:
> >
> > Boca BB1004 4-Port serial card (Modems NOT supported)
> > Boca IOAT66 6-Port serial card (Modems supported)
> > Boca BB1008 8-Port serial card (Modems NOT supported)
> > Boca BB2016 16-Port serial card (Modems supported)
>
> That is reasonable.
>
> I have been toying with the idea of picking up an IOAT66, but I've had
> bad experiences with a BB2016 (interrupt problems) and I am a bit
> hesitant to try a Boca serial board again. :-) I guess that's fine
> since I don't think I know anyone who sells them anyways. But they
> ARE very inexpensive!
Ahhh... I sell them.... just not a regular item :-)
- --
Rod Grimes rgrimes@gndrsh.aac.dev.com
Accurate Automation, Inc. Reliable computers for FreeBSD
------------------------------
From: Terry Lambert <terry@lambert.org>
Date: Fri, 15 Nov 1996 14:06:50 -0700 (MST)
Subject: Re: Sockets question...
> >How do I read into a structure on a machine that demands aligned
> >data access?
>
> You read into an intermediate buffer and copy it. You have to convert from
> network to machine representation anyway, so this isn't (much) more overhead.
DCE RPC does not do a conversion if both ends are the same byte order.
BSD RPC does not do a conversion on hardware with correct endianess.
I think this is why #pragma pack was invented. 8-(.
> >nothing would work at all if you couldn't issue a read for n
> >bytes that didn't complete until you *got* n bytes.
>
> Well, I guess the BSD networking code has probably never worked at all.
> The read() system call on a socket is based on soreceive(), which
> returns up to N bytes.
On a non-blocking socket, right?
Or in the case of a blocking call for more bytes than SO_RECVBUF (which
you can determine via getsockopt and therefore never trigger).
Can it return less than N otherwise? The WAITALL case is only for the
user buffer larger than SO_RECVBUF, right?
Terry Lambert
terry@lambert.org
- ---
Any opinions in this posting are my own and not those of my present
or previous employers.
------------------------------
From: Terry Lambert <terry@lambert.org>
Date: Fri, 15 Nov 1996 14:26:41 -0700 (MST)
Subject: Re: Sockets question...
> > Otherwise, you have just un-formatted your transport contents. 8-(.
>
> TCP streams are byte oriented, not record oriented. There is no preservation
> of record boundaries with TCP.
I am talking about imposing structure on the stream on the basis of
forcing the programs to communicate with the stream using record sized
chunks. It's irrelevent what happens in the wire, or how the kernel
chooses to copy the byte-oriented data into the user buffer on the
receiver, so long as the receiver is not falsly notified that the
request has been fulfilled.
I admit that there is an implication here that I didn't take into
account: That two writes to a socket will not be agregated across
write buffer boundries into a single packet, OR that the receiver
will read an amount such that if agregation occurs, it is irrelevent
to the receiver's ability to distinguish the boundries.
I have to say that SO_RCVLOWAT and SO_RCVTIMEO screw this considerably,
in their roles as vmin/vtime equivalents. 8-(. My advice for someone
building a transaction oriented protocol would be to set SO_RCVLOWAT to
the same value as SO_RCVBUF, and to never, ever do packets larger than
that as part of the formatted data.
> > If you want to get technical, according to this description, if you are
> > using a SOCK_STREAM, then a read on a blocking socket will act like a
> > recv(2) or recvfrom(2) with flags MSG_WAITALL by default.
>
> read(2), recv(2), and recvfrom(2) all call soreceive() for sockets. read(2)
> does NOT set MSG_WAITALL when calling soreceive(). Also:
Well, this is probably an error for a blocking socket read, then... 8-(.
> * If MSG_WAITALL is set but resid is larger than the receive buffer,
> * we have to do the receive in sections, and thus risk returning
> * a short count if a timeout or signal occurs after we start.
>
> So MSG_WAITALL can also return a short count.
Only in case of a timeout or a signal, both of which are error
conditions. The default behaviour for BSD prior to the introduction
of siginterrupt() was to restart all system calls interrupted by signal.
If the default behaviour is used, then the only possible failure
would be EPIPE, indicating a timeout (loss of connection) (assuming
a blocking socket).
> > Maybe you should be using SOCK_SEQPACKET instead of SOCK_STREAM?
>
> That might be difficult.
>
> A SOCK_SEQPACKET socket [ .... ] is protocol specific, and presently
> implemented only for PF_NS. (PF_NS == Xerox Network Systems protocols)
Well, bummer deal. 8-(.
Terry Lambert
terry@lambert.org
- ---
Any opinions in this posting are my own and not those of my present
or previous employers.
------------------------------
From: Karl Denninger <karl@Mcs.Net>
Date: Fri, 15 Nov 1996 15:46:43 -0600 (CST)
Subject: Re: Sockets question...
> > No; what happens for you at the receiving end is that the packets get
> > reassembled into the same *stream* of data as they were at the sender.
> > The sender can buffer small writes into big chunks, or can fragment big
> > writes. If the underlying MTU of the path changes, writes that didn't
> > used to have to be fragmented may now have to be. Packet N can get
> > lost in the network, while packets N+1, N+2, N+3 and N+4 arrive; when
> > the sender retransmits packet N all of a sudden a ton of data "shows
> > up" at the receiver. The receiver's job is *solely* to put those
> > packets back together into the same stream as the sender sent.
>
> So at the "read" interface, you *can* count on it arriving in the
> same sized chunks as you wrote it, as long as the order and chunk size of
> your reads is identical to those of your writes... in other words,
> synchronized client and server automatons.
>
> Right?
>
> IF I issue a blocking read for 1000 bytes, AND
> I issue a blocking write for 1000 bytes, AND
> the blocking write returns "1000",
> THEN the blocking read will return 1000 bytes OR
> it won't return (EPIPE after connection timeout).
>
> Karl is saying he has synchronized client and server automatons, but
> that it's not working for large buffer sizes on the writes.
No, Karl is doing this:
1) The *writer* is writing records of variable size with a prefix to
indicate how many byte(s) follow.
2) The writer does this ASSUMING that all of the records will get
delivered to the reader.
3) When the writer is done, he writes a "no more records follow"
flag record.
4) All of those writes return with no errors.
5) The READER gets about 2700 of the records (out of 8500!) and NEVER
SEES ANY MORE DATA. It hangs in read()!
This does NOT happen with the 2.6.3 development kit and libraries. It
RELIABLY happens with -current.
- --
- --
Karl Denninger (karl@MCS.Net)| MCSNet - The Finest Internet Connectivity
http://www.mcs.net/~karl | T1's from $600 monthly to FULL DS-3 Service
| 33 Analog Prefixes, 13 ISDN, Web servers $75/mo
Voice: [+1 312 803-MCS1 x219]| Email to "info@mcs.net" WWW: http://www.mcs.net/
Fax: [+1 312 248-9865] | 2 FULL DS-3 Internet links; 400Mbps B/W Internal
------------------------------
From: Bill Fenner <fenner@parc.xerox.com>
Date: Fri, 15 Nov 1996 13:54:58 PST
Subject: Re: Sockets question...
Terry:
>Bill:
> Terry:
>> >So at the "read" interface, you *can* count on it arriving in the
>> >same sized chunks as you wrote it
>>
>> No, you can *never* count on that, since non-blocking reads from a stream
>> socket return as much data as is available, which could be less than you
>> asked for. See soo_read() (or soo_rw() in earlier BSD's) and soreceive().
>
>By default, the sockets are blocking. You have to go out of your
>way to make them non-blocking (ie: loading the gun before you can shoot
>yourself in the foot).
Terry, STOP SPECULATING. Look at the code. soo_read() is what is called
by the read system call, and it calls soreceive(). soreceive() returns
whatever is waiting, up to what is requested, unless you set MSG_WAITALL.
soo_read() doesn't set MSG_WAITALL, so the semantics of read() on a
blocking socket is to return as much data is waiting, up to the amount
that was requested.
>But on a blocking socket, it doesn't make sense to have to issue multiple
>system calls to read chunks of a whole message when you aren't going to
>do anything with it until all the reads have been satisfied?
The CSRG apparently felt otherwise.
Bill
------------------------------
From: Bill Fenner <fenner@parc.xerox.com>
Date: Fri, 15 Nov 1996 13:58:48 PST
Subject: Re: Sockets question...
In message <199611152146.PAA10779@Jupiter.Mcs.Net> Karl wrote:
>This does NOT happen with the 2.6.3 development kit and libraries. It
>RELIABLY happens with -current.
This points clearly at:
a) incorrect code, compiled one way by 2.6.3 and differently by 2.7.2
b) a compiler bug.
We should all stop arguing about SOCK_STREAM semantics.
Bill
------------------------------
End of hackers-digest V1 #1645
******************************
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199611152201.OAA20062>