inpcb: fully retire inp_ppcb pointer

Before a protocol specific control block started to embed inpcb in self
(see 0aa120d52f3c, e68b3792440c, 483fe96511ec) this pointer used to point
at it.

Retain kf_sock_inpcb field in the struct kinfo_file in <sys/user.h>. The
exp-run detected a minimal use of the field in ports:
* sysutils/lsof - patched upstream
* net-mgmt/netdata - patch accepted upstream
* emulators/qemu-user-static - upstream master branch seems not using
the field anymore
We can keep the field around for some time, but eventually it may be
reused for something else.

PR: 277659 (exp-run)
Reviewed by: tuexen
Differential Revision: https://reviews.freebsd.org/D44491

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: two-line .c pattern

Remove /^#include\s+<sys/cdefs.h>.*$\n\s+__FBSDID\("\$FreeBSD\$"\);\n/

tcp: retire TCPDEBUG

This subsystem is superseded by modern debugging facilities,
e.g. DTrace probes and TCP black box logging.

We intentionally leave SO_DEBUG in place, as many utilities may
set it on a socket. Also the tcp::debug DTrace probes look at
this flag on a socket.

Reviewed by: gnn, tuexen
Discussed with: rscheff, rrs, jtl
Differential revision: https://reviews.freebsd.org/D37694

tcp: embed inpcb into tcpcb

For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.

The most import one is the inpcb itself. With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa. Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that. However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort. The new intotcpcb() macro is ready for such move.

The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.

One interesting side effect is that now TCP data is allocated from
SMR-protected zone. Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.

Large part of the change was done with sed script:

s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g

Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.

Differential revision: https://reviews.freebsd.org/D37127

systat: remove INP_TIMEWAIT

systat: clean up code assuming network classes

Similar to netstat, clean up code that uses inet_lnaof() to check for
binding to "host 0" (lowest host on network) as a "network" bind.
Such things don't happen, and current networks are seldom if ever
found in /etc/networks.

MFC after: 1 month
Reviewers: tuexen
Differential Revision: https://reviews.freebsd.org/D32720

systat: mostly clean up warns

Sponsored by: Rubicon Communications, LLC ("Netgate")

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Hide struct socket and struct unpcb from the userland.

Violators may define _WANT_SOCKET and _WANT_UNPCB respectively and
are not guaranteed for stability of the structures. The violators
list is the the usual one: libprocstat(3) and netstat(1) internally
and lsof in ports.

In struct xunpcb remove the inclusion of kernel structure and add
a bunch of spare fields. The xsocket already has socket not included,
but add there spares as well. Embed xsockbuf into xsocket.

Sort declarations in sys/socketvar.h to separate kernel only from
userland available ones.

PR: 221820 (exp-run)

Hide struct inpcb, struct tcpcb from the userland.

This is a painful change, but it is needed. On the one hand, we avoid
modifying them, and this slows down some ideas, on the other hand we still
eventually modify them and tools like netstat(1) never work on next version of
FreeBSD. We maintain a ton of spares in them, and we already got some ifdef
hell at the end of tcpcb.

Details:
- Hide struct inpcb, struct tcpcb under _KERNEL || _WANT_FOO.
- Make struct xinpcb, struct xtcpcb pure API structures, not including
kernel structures inpcb and tcpcb inside. Export into these structures
the fields from inpcb and tcpcb that are known to be used, and put there
a ton of spare space.
- Make kernel and userland utilities compilable after these changes.
- Bump __FreeBSD_version.

Reviewed by: rrs, gnn
Differential Revision: D10018

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Use NULL instead of 0 for pointers.

gethostbyname(3) will return NULL for error status.

MFC after: 2 weeks.

Augment struct tcpstat with tcps_states[], which is used for book-keeping
the amount of TCP connections by state. Provides a cheap way to get
connection count without traversing the whole pcb list.

Sponsored by: Netflix

- Avoid accessing window properties directly, instead, use accessors.
This should be no-op for now, but allows the code to work if we
move to NCURSES_OPAQUE.
- Use calloc() instead of malloc+bzero.

MFC after: 2 weeks

Merge from projects/sendfile:

o Introduce a notion of "not ready" mbufs in socket buffers. These
mbufs are now being populated by some I/O in background and are
referenced outside. This forces following implications:
- An mbuf which is "not ready" can't be taken out of the buffer.
- An mbuf that is behind a "not ready" in the queue neither.
- If sockbet buffer is flushed, then "not ready" mbufs shouln't be
freed.

o In struct sockbuf the sb_cc field is split into sb_ccc and sb_acc.
The sb_ccc stands for ""claimed character count", or "committed
character count". And the sb_acc is "available character count".
Consumers of socket buffer API shouldn't already access them directly,
but use sbused() and sbavail() respectively.
o Not ready mbufs are marked with M_NOTREADY, and ready but blocked ones
with M_BLOCKED.
o New field sb_fnrdy points to the first not ready mbuf, to avoid linear
search.
o New function sbready() is provided to activate certain amount of mbufs
in a socket buffer.

A special note on SCTP:
SCTP has its own sockbufs. Unfortunately, FreeBSD stack doesn't yet
allow protocol specific sockbufs. Thus, SCTP does some hacks to make
itself compatible with FreeBSD: it manages sockbufs on its own, but keeps
sb_cc updated to inform the stack of amount of data in them. The new
notion of "not ready" data isn't supported by SCTP. Instead, only a
mechanical substitute is done: s/sb_cc/sb_ccc/.
A proper solution would be to take away struct sockbuf from struct
socket and allow protocols to implement their own socket buffers, like
SCTP already does. This was discussed with rrs@.

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Whitespace fixes.

- Remove redundant empty lines.
- Replace ^L by \014. This allows you to safely cat/grep/etc this file
without causing confusion.

Replace index() and rindex() calls with strchr() and strrchr().

The index() and rindex() functions were marked LEGACY in the 2001
revision of POSIX and were subsequently removed from the 2008 revision.
The strchr() and strrchr() functions are part of the C standard.

This makes the source code a lot more consistent, as most of these C
files also call into other str*() routines. In fact, about a dozen
already perform strchr() calls.

Fix whitespace inconsistencies in systat(1).

According to md5(1), the resulting binary is the same.

Remove duplicated header files

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Revert most part of 200420 as requested, as more review and polish is
needed.

Remove unneeded header includes from usr.bin/ except contributed code.

Tested with: make universe

Correct a number of evolved problems with inp_vflag and inp_flags:
certain flags that should have been in inp_flags ended up in inp_vflag,
meaning that they were inconsistently locked, and in one case,
interpreted. Move the following flags from inp_vflag to gaps in the
inp_flags space (and clean up the inp_flags constants to make gaps
more obvious to future takers):

INP_TIMEWAIT
INP_SOCKREF
INP_ONESBCAST
INP_DROPPED

Some aspects of this change have no effect on kernel ABI at all, as these
are UDP/TCP/IP-internal uses; however, netstat and sockstat detect
INP_TIMEWAIT when listing TCP sockets, so any MFC will need to take this
into account.

MFC after: 1 week (or after dependencies are MFC'd)
Reviewed by: bz

- Import the HEAD csup code which is the basis for the cvsmode work.

ANSIfy and remove register.

Resulting binary verified with strip(1)+md5(1).

The logic of fetchnetstat_sysctl() isn't too complex: if idx is 0,
we set and use xtp; if idx is 1, we set and use xip; the other cases
are impossible. However, GCC cannot see that xip and xtp are always
initialized before use because they are initialized and used in
different if/else blocks. So setting them to NULL at the very
beginning won't hurt.

+ WARNS=4 reminds that nlist.n_name isn't const.
+ Use C99 initializers to be WARNS-clean.
+ The last element in a namelist should have its n_name set to NULL,
not to an empty string.

Don't discard a const qualifier from constant strings.

Remove trailing whitespace.

Submitted by: Se=E1n Farley <sean-freebsd at farley dot org>
PR: bin/81874

Show the load average in the tcp display (it was already shown, perhaps
not very usefully, in all other displays). This was the original point
of the PR.

Move the load average up by 2 so that it starts in row 0 for all windows
(2 lines above it were wasted for all other windows except vmstat).
Move everything below it up by 2 or 3 (3 for icmp and icmp6 which had
an extra blank line due from not compensating for the foot-shooting in
note (3); only ip and ip6 compensated). Reduce the magic numbers related
to this.

Notes by the submitter:
%%%
1. All the subwin() calls are identical using #define MAINWIN_ROW 3
(systat.h).
2. The load average is at the top of the window.
3. Each display starts on the fourth line. I made changes to those
displays that shifted the start line (i.e., icmp). This entailed a
lot of changes within the comments at the top of those displays.
4. For ip6, I shifted the "Input next-header histogram" column down one
row to separate it from "IPv6 Output". I raised "bad scope packets"
and "address selection failed" up one row to stay with "IPv6 Input"
(valid?). They were down one row to probably line up at the bottom,
but I think they should stick with their fellow items in a column.
5. I condensed ifstat a bit. It had a lot of empty rows.
%%%

Submitted by: Se=E1n Farley <sean-freebsd at farley dot org>
PR: bin/81874

simply ignore unknown address family.

MFC after: 1 week

teach IPv6 to `systat -netstat'.

Tested by: kuriyama
MFC after: 1 week

Make systat -net aware of compressed time_wait sockets.

Add ip6 and icmp6 displays to systat.

MFC after: 2 weeks

Widen struct sockbuf's sb_timeo member to int from short. With
non-default but reasonable values of hz this member overflowed,
breaking NFS over UDP.

Also, as long as I'm plowing up struct sockbuf ... Change certain
members from u_long/long to u_int/int in order to reduce wasted
space on 64-bit machines. This change was requested by Andrew
Gallatin.

Netstat and systat need to be rebuilt. I am incrementing
__FreeBSD_version in case any ports need to change.

remove __P

WARNS=2 fixes with NO_WERROR set, as there are some header issues
with namelists. use __FBSDID().

Get rid of setgid kmem for systat, and while being there, fix some bugs
and compiler warnings.
The data for network statistics are still obtained via the kvm interface
if systat was started with the needed privileges, otherwise sysctls are
used. The reason for this is that with really many open sockets, the
sysctl method is probably slower, but it systat -netstat is probably not
really usable in either mode under these conditions.

Approved by: rwatson

My bad, committed the submitted patch rather than the fixed patch.

Use macro API to <sys/queue.h>

Submitted by: "Jason" <jsmethers@pdq.net>
Reviewed by: phk

systat is way too comfortable with curses internals...

$Id$ -> $FreeBSD$

Quiet many compiler warnings.

#include <arpa/inet.h>

o Use snprintf rather than sprintf
o Add more checks for buffer overflows
o Use snprintf rather than strcat/cpy and have better checks for max
length exceeded.

Most of these changes are not exploitable buffer overruns, but it never
hurts to be safe.

Inspired by and obtained from: OpenBSD

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

bin/1942 curses problem in systat
truncate if too long for field.
2.2 candidate.

Eliminate unnecessary include of <sys/mbuf.h>.

Move #include of queue.h before #include of socketvar.h in preparation for
struct socket changes.

Remove trailing whitespace.

Updated for 4.4BSD queue macros. (Oops, I forgot to commit this last night;
sorry Jordan).

BSD 4.4 Lite Usr.bin Sources