usr.bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

Remove my middle name.

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

sockstat(1): print PID adjusted to the right

This allows for easy copy-and-paste of a unix(4) peer name for lookup.
With current implementation it is guaranteed that a peer listed could be
found in the output.

Differential revision: https://reviews.freebsd.org/D35727

sockstat(1): print out full connection graph for unix(4) sockets

Kernel provides us with enough information to display all possible
connections between UNIX sockets.

o Store unp_conn, xu_firstref and xu_nextref in the faddr of a UNIX sock.
o Build tree of file descriptors, indexed by the socket pointer.
o In displaysock() print out all possible information:
1) if socket is bound, print name of this socket
2) if socket has connected to a peer with a name, print peers name
3) if socket has connected to a peer without a name, print [pid fd]
4) if a bound socket has received connections, print list of them
as [pid fd]
Previously, only 1) either 2) were printed.

Reviewed by: tuexen
Differential revision: https://reviews.freebsd.org/D35726

sockstat(1): widen the FD printing format

Three digit file descriptors are very common these days. So let the
columns not break with them.

Reviewed by: tuexen
Differential revision: https://reviews.freebsd.org/D35725

sockstat(1): use tree(3) rbtree instead of hash

o Use tree to lookup by socket kvaddr. The size of hash is too big for a
small virtual machine and at the same time too little for a large
production server. A tree would better fit here.
o For those pcbs, that don't have a socket associated, use a list.
o Provide a second tree to lookup by pcb kvaddr. These removes full hash
traversal when printing every unix(4) socket.

Reviewed by: tuexen
Differential revision: https://reviews.freebsd.org/D35724

sockstat(1): we don't need kernel struct unpcb

Fixes: 0e229f343f4e8791f84a7acc6300f0bd1476c7b5

sockstat: be more verbose when reporting kernel/userland size mismatch.

MFC after: 2 weeks

sockstat: fix usage string

MFC after: 1 week
Sponsored by: Netflix, Inc.

sockstat: add -i to display inp_gencnt

The inp_gencnt will be used to identify a TCP endpoint by an upcoming
command line tool to set TCP socket options.

Reviewed by: rscheff
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D34137

sockstat: change check for wildcard sockets to avoid historical classes

sockstat was checking whether a bound address was "host 0", the lowest
host on a network, using inet_lnaof(). This only works for class A/B/C.
However, it isn't useful to bind such an address unless it is really
the unspecified address INADDR_ANY. Change the check to to use that.

MFC after: 1 month
Reviewd by: tuexen
Differential Revision: https://reviews.freebsd.org/D32715

sockstat: don't query jail vnet if system is running without VIMAGE.

Fixes: f1cd4902bf17

sockstat: make err(3) on jail errors more verbose.

sockstat: use cap_pwd

The sockstat is using password database operations to obtain
the username. Such operations are disallowed in capability mode.
For such operations Casper is required.

Reported by: olivier@
Tested by: olivier@

sockstat: fix core dump

The get_proto_type function is used before creating the Casper function.
This is why we have to distinguish when the Casper service is allocated or not.

Reported by: olivier@
Tested by: olivier@

sockstat: Use libcasper to capsicumize

Drop rights we do not need. This has to be done after jail_attach.

Reviewed by: oshogbo
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D26958

tcp: add support for TCP over UDP

Adding support for TCP over UDP allows communication with
TCP stacks which can be implemented in userspace without
requiring special priviledges or specific support by the OS.
This is joint work with rrs.

Reviewed by: rrs
Sponsored by: Netflix, Inc.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D29469

Add "-n" flag to sockstat.

sockstat can "hang" on getpwuid() calls in situations when FreeBSD
is joined to a directory service (AD/LDAP etc) and the directory
service fail to answer in a timely manner when trying to resolve
numeric UIDs to user names.

Submitted by: Caleb St. John <caleb@ixsystems.com>
MFC after: 1 week

sockstat: Fix error message when jail_attach fails

jail_errmsg is for libjail, jail_attach() is a system call.

Add a -C option to sockstat to display the congestion control for TCP
connections.

Reviewed by: rscheff
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D26413

sockstat: Attach to jail if in new vnet

Attach sockstat -j to the specified jail if the jail is in a new vnet.
Otherwise we do not see all sockets belonging to the jail.

Reviewed by: jamie
Approved by: mmacy (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24413

Make struct xinpcb and friends word-size independent.

Replace size_t members with ksize_t (uint64_t) and pointer members
(never used as pointers in userspace, but instead as unique
idenitifiers) with kvaddr_t (uint64_t). This makes the structs
identical between 32-bit and 64-bit ABIs.

On 64-bit bit systems, the ABI is maintained. On 32-bit systems,
this is an ABI breaking change. The ABI of most of these structs
was previously broken in r315662. This also imposes a small API
change on userspace consumers who must handle kernel pointers
becoming virtual addresses.

PR: 228301 (exp-run by antoine)
Reviewed by: jtl, kib, rwatson (various versions)
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15386

Allow jail names (not just IDs) to be specified for: cpuset(1), ipfw(8),
sockstat(1), ugidfw(8)
These are the last of the jail-aware userland utilities that didn't work
with names.

PR: 229266
MFC after: 3 days
Differential Revision: D16047

sockstat: add break that was forgot in 328279

Reported by: garga@
MFC after: 1 week
X-MFC With: 328279
Sponsored by: Gandi.net

sockstat: Add -q option to suppress the header line

MFC after: 1 week
Sponsored by: Gandi.net

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Hide struct socket and struct unpcb from the userland.

Violators may define _WANT_SOCKET and _WANT_UNPCB respectively and
are not guaranteed for stability of the structures. The violators
list is the the usual one: libprocstat(3) and netstat(1) internally
and lsof in ports.

In struct xunpcb remove the inclusion of kernel structure and add
a bunch of spare fields. The xsocket already has socket not included,
but add there spares as well. Embed xsockbuf into xsocket.

Sort declarations in sys/socketvar.h to separate kernel only from
userland available ones.

PR: 221820 (exp-run)

Remove tailing whitespace.

Add a command line option for using a wider field for displaying
addresses. This allows the table to be consistent when IPv6
addresses have to be printed.
While there, document the -v option in the man page.

Sponsored by: Netflix, Inc.

Add support for printing the path state for SCTP association.

Add printing of the remote encapsulation port for SCTP associations.

Sponsored by: Netflix, Inc.

Add support to print the TCP stack being used.

Sponsored by: Netflix, Inc.

Shut up gcc.

Hide struct inpcb, struct tcpcb from the userland.

This is a painful change, but it is needed. On the one hand, we avoid
modifying them, and this slows down some ideas, on the other hand we still
eventually modify them and tools like netstat(1) never work on next version of
FreeBSD. We maintain a ton of spares in them, and we already got some ifdef
hell at the end of tcpcb.

Details:
- Hide struct inpcb, struct tcpcb under _KERNEL || _WANT_FOO.
- Make struct xinpcb, struct xtcpcb pure API structures, not including
kernel structures inpcb and tcpcb inside. Export into these structures
the fields from inpcb and tcpcb that are known to be used, and put there
a ton of spare space.
- Make kernel and userland utilities compilable after these changes.
- Bump __FreeBSD_version.

Reviewed by: rrs, gnn
Differential Revision: D10018

Fix style(9)

Reviewed by: ngie, tuexen, vangyzen, allanjude
Approved by: allanjude
MFC after: 1 week
Sponsored by: Rubicon Communications (Netgate)
Differential Revision: https://reviews.freebsd.org/D9588

This patch fixes two bugs:
* sctp46, tcp46, and udp46 sockets are displayed as such and not as
sctp4 6, tcp4 6, udp4 6. This bug was introduced in
http://svnweb.freebsd.org/base?view=revision&revision=187915
* For SCTP sockets, the the -4 and -6 flags are honoured as much as
possible. This means IPv4 sockets are handled correctly, IPv6
sockets are displayed as sctp46, since it is currently not possible
to distinguish between sctp6 and sctp46.

Approved by: re (gjb)
MFC after: 1 week

Fix two types which resulted in setting the address long wrong
for IPv6 addresses.

Reported by: pfg@
CID: 1347086
MFC after: 1 week

Fix a ton of speelling errors

arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337

Report CLOSED as state for bound sockets for consistency with
netstat and the way TCP is handled.

PR: 201585
MFC after: 3 weeks

Add SCTP support.

PR: 201585
MFC after: 3 weeks

Remove a variable which is set but never used.

PR: 201585
MFC after: 3 weeks

Print the status on the first line of a socket description, not at
the last one. This doesn't matter for TCP, but it does for the upcoming
SCTP support.

PR: 201585
MFC after: 3 weeks

Move assignments around to avoid a false-positive uninitialized variable
warning which broke the sparc64 build.

PR: 201585
MFC after: 3 weeks

Add a -s option which adds a column listing the connection state if
applicable (currently only for TCP).

PR: 201585
MFC after: 3 weeks

- Add SOCK_SEQPACKET support in UNIX-domain socket.
- Display zoneid using % notation in an IPv6 address.
- Use nitems().
- Use sstos{in,in6,un} macros to simplify casts.
- style(9).

Don't leak sockets.

Reported by: Coverity
CID: 1306785

MFC after: 3 days

Fix a bug reported by coverity. Since AF_UNIX sockets don't
have multiple addresses, the problem didn't show up during testing.

Reported by: Coverity
CID: 1306787

Add support for SCTP.

MFC after: 3 days

Allow more than one local or remote address per socket. This is needed to
support SCTP (and MPTCP in the future). No functional change for existing
protocols.

MFC after: 3 days

Allow more than one socket entry for a file descriptor. This is needed
for supporting 1-to-many style SCTP sockets. For other sochets, there
is no functional change.

MFC after: 3 days

Simplify printaddr(). No functional change.

MFC after: 3 days

Update the usage with the new jail option.

Spotted by: Jason Hellenthal
MFC after: 3 days

- Updated TOE support in the kernel.

- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
These are available as t3_tom and t4_tom modules that augment cxgb(4)
and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as
usual with or without these extra features.

- iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the
works and will follow soon.

Build-tested with make universe.

30s overview
============
What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE

Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe

Which connections are offloaded? Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe

Reviewed by: bz, gnn
Sponsored by: Chelsio communications.
MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)

Allow the socket list to be limited to a specific jail id.

No objections: current@

Try to avoid ambiguity when sysctl returns ENOMEM additionally
checking the returned oldlen: when ENOMEM is due to the supplied
buffer being too short the return oldlen is equal to buffer size.

Without this additional check sockstat gets stuck in loop leaking the
memory if the returned ENOMEM was due the exceeded memorylocked
limit. This is easily can be observed running `limits -l 1k sockstat'.

Submitted by: Andrey Zonov <andrey zonov org>
MFC after: 1 week

sockstat: Also show sockets not associated with a file descriptor.

Sockets not associated with a file descriptor include TCP TIME_WAIT states
and sockets created via the socket(9) API such as from rpc.lockd and the NFS
client.

PR: bin/164081
MFC after: 2 weeks
No objection: des

Reencode files from latin1 to UTF-8.

This makes a tiny percentage of entries in calendars ugly for latin1
users, but fixes them for UTF-8 users.

This badly needs a solution involving locale-dependent re-encoding.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFC r205874: make sockstat -6 output more readable for long ipv6
addresses.

o Make sockstat -6 output more readable for long ipv6
addresses (most of them apart from ::1): put a whitespace
between local and remote address:port pairs.

PR: bin/145194
Submitted by: Fedor Dikarev
MFC after: 2 weeks

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Mute some warnings on uninitialized variables.

The code does the right thing, but the compiler is unable to figure it
out. All paths that use that variable use the same invariant.

Remove unused v6 macro aliases for inpcb fields:

in6p_ip6_nxt
in6p_vflag
in6p_flags
in6p_socket
in6p_lport
in6p_fport
in6p_ppcb

Remove unused v6 macro aliases for inpcb flags:

IN6P_HIGHPORT
IN6P_LOWPORT
IN6P_ANONPORT
IN6P_RECVIF
IN6P_MTUDISC
IN6P_FAITH
IN6P_CONTROLOPTS

References to in6p_lport and in6_fport in sockstat are also replaced with
normal inp_lport and inp_fport references.

MFC after: 3 days
Reviewed by: bz

Make sure the entries don't run into each other when they're longer than
the allotted space.

PR: bin/129318
Submitted by: Ighighi <ighighi@gmail.com>
MFC after: 3 weeks

- Import the HEAD csup code which is the basis for the cvsmode work.

Add -L to usage().

Add an -L option to ignore loopback Internet sockets.

MFC after: 2 weeks

o Do not warn if the process exits before we get its name.

PR: bin/113777
Submitted by: Dmitrij Tejblum
MFC after: 1 week

Use proc name (ki_comm) instead of thread name (ki_ocomm) as these may now
be different.

In revision 1.14 I broke the -4 and -6 options of sockstat(1).
Using either one of the two would result in an empty protos[]
array, and no sockets were actually listed:

% sockstat -4
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
% sockstat -6
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
%

Fix this bug by tweaking appropriately the logic of handling opt_4,
opt_6, opt_u and protos_defined.

Submitted by: des
Pointy hat: keramida

Add support for filtering sockets by protocol type. The default
behavior of sockstat(1) will still be to show "udp", "tcp" and
"divert" protocols, but we can now provide a (comma-separated)
list of protocols, as in:

% sockstat -P tcp

to list only TCP sockets, or we can filter more than one protocol
by separating the protocol names with a comma:

% sockstat -P tcp,udp

Protocol names are parsed with getprotobyname(3), so any protocol
whose name is listed in `/etc/protocols' should work fine.

Submitted by: Josh Carroll <josh.carroll@psualum.com>
Approved by: des

Don't free(sock) before it's even allocated.

Submitted by: Joerg Sonnenberger <joerg@britannica.bec.de>
MFC after: 1 week

Unbreak sockstat(1) on systems without the divert protocol.

Show divert(4) sockets as well.

Do not display bogus entries for sockets in the TIME_WAIT or similar
states that no longer have a corresponding file descriptor - until now,
sockstat would mostly randomly match null kern.file.*.xf_data fields
with the first mostly-closed socket.

This bugfix is a RELENG_5 candidate.

Approved by: andre

make sockstat not print wierd addresses on not connected unix domain sockets

Pointed out by: rwatson
Reviewed by: peter

Finish the implementation of the -p switch so that it
actually works. I have no idea why this wasn't finished
and happened to try to use it.

To reserve space for 65536 bits, allocate
65536 / (sizeof(int) * CHAR_BITS) `int's instead of
65536 / (sizeof(int) * CHAR_BITS) bytes to avoid a possible
segmentation fault if ports above 16383 are specified via the
-p option on a platform with 4 byte wide ints.

Approved by: re (bmah)
Reported by: Marco Wertejuk <wertejuk@mwcis.com>

Bow to the whining masses and change a union back into void *. Retain
removal of unnecessary casts and throw in some minor cleanups to see if
anyone complains, just for the hell of it.

Change struct file f_data to un_data, a union of the correct struct
pointer types, and remove a huge number of casts from code using it.

Change struct xfile xf_data to xun_data (ABI is still compatible).

If we need to add a #define for f_data and xf_data we can, but I don't
think it will be necessary. There are no operational changes in this
commit.

Use a cast to `void *' before casting to a pointer to a structure
to stop GCC emitting warnings about increased alignment requirements
which broke the build for sparc64.

Approved by: des

Don't depend on namespace pollution from <netinet/in_pcb.h>.

Submitted by: bde

Implement the -l and -c options, which I'd forgotten.

Rewrite sockstat(1) in C.

Sponsored by: DARPA, NAI Labs