inpcb: fully retire inp_ppcb pointer

Before a protocol specific control block started to embed inpcb in self
(see 0aa120d52f3c, e68b3792440c, 483fe96511ec) this pointer used to point
at it.

Retain kf_sock_inpcb field in the struct kinfo_file in <sys/user.h>. The
exp-run detected a minimal use of the field in ports:
* sysutils/lsof - patched upstream
* net-mgmt/netdata - patch accepted upstream
* emulators/qemu-user-static - upstream master branch seems not using
the field anymore
We can keep the field around for some time, but eventually it may be
reused for something else.

PR: 277659 (exp-run)
Reviewed by: tuexen
Differential Revision: https://reviews.freebsd.org/D44491

usr.bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Add the fstat -s option to display socket information.

Reviewed by: jilles
MFC after: 1 week
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D21880

Print type designator 'D' for the KF_TYPE_DEV files.

No type-specific data is provided by the kernel.

Sponsored by: Mellanox Technologies
MFC after: 1 week

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

fstat: catch up with r318997 and use 64 bits to store fsid

Discussed with: kib

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Similar to 233760 and 236717, export some more useful info about the
kernel-based POSIX semaphore descriptors to userland via procstat(1) and
fstat(1):
- Change sem file descriptors to track the pathname they are associated
with and add a ksem_info() method to copy the path out to a
caller-supplied buffer.
- Use the fo_stat() method of shared memory objects and ksem_info() to
export the path, mode, and value of a semaphore via struct kinfo_file.
- Add a struct semstat to the libprocstat(3) interface along with a
procstat_get_sem_info() to export the mode and value of a semaphore.
- Teach fstat about semaphores and to display their path, mode, and value.

MFC after: 2 weeks

Don't cast inode number or file size down to long or unsigned.

Since ino_t size is about to change to 64-bits, casts to long would
truncate 64-bit numbers on 32-bit archs.

Sponsored by: Google Summer of Code 2011

Export some more useful info about shared memory objects to userland
via procstat(1) and fstat(1):
- Change shm file descriptors to track the pathname they are associated
with and add a shm_path() method to copy the path out to a caller-supplied
buffer.
- Use the fo_stat() method of shared memory objects and shm_path() to
export the path, mode, and size of a shared memory object via
struct kinfo_file.
- Add a struct shmstat to the libprocstat(3) interface along with a
procstat_get_shm_info() to export the mode and size of a shared memory
object.
- Change procstat to always print out the path for a given object if it
is valid.
- Teach fstat about shared memory objects and to display their path,
mode, and size.

MFC after: 2 weeks

Backout r230934 which didn't work with unix sockets and
several filesystem layers mounted at the specified path.

Pointy hat to: pluknet

Print the owner process for unix domain sockets when restricted to the
specified files.

PR: bin/143962
MFC after: 2 weeks

Spelling fixes for usr.bin/

Mark global functions and/or variables in fstat(1) static where possible.

This allows compilers and static analyzers to more thorough analysis.

Correct column with for device numbers made in previous change.

The device number should be displayed using only five columns -- not eight.

Get rid of major/minor number distinction.

As of FreeBSD 6, devices can only be opened through devfs. These device
nodes don't have major and minor numbers anymore. The st_rdev field in
struct stat is simply based a copy of st_ino.

Simply display device numbers as hexadecimal, using "%#jx". This is
allowed by POSIX, since it explicitly states things like the following
(example taken from ls(1)):

"If the file is a character special or block special file, the
size of the file may be replaced with implementation-defined
information associated with the device in question."

This makes the output of these commands more compact. For example, ls(1)
now uses approximately four columns less. While there, simplify the
column length calculation from ls(1) by calling snprintf() with a NULL
buffer.

Don't be afraid; if needed one can still obtain individual major/minor
numbers using stat(1).

uintmax_t is a better type here...

Submitted by: avg@

Fix printf int mismatch on 32-bit architectures...

- Commit work from libprocstat project. These patches add support for runtime
file and processes information retrieval from the running kernel via sysctl
in the form of new library, libprocstat. The library also supports KVM backend
for analyzing memory crash dumps. Both procstat(1) and fstat(1) utilities have
been modified to take advantage of the library (as the bonus point the fstat(1)
utility no longer need superuser privileges to operate), and the procstat(1)
utility is now able to display information from memory dumps as well.

The newly introduced fuser(1) utility also uses this library and able to operate
via sysctl and kvm backends.

The library is by no means complete (e.g. KVM backend is missing vnode name
resolution routines, and there're no manpages for the library itself) so I
plan to improve it further. I'm commiting it so it will get wider exposure
and review.

We won't be able to MFC this work as it relies on changes in HEAD, which
was introduced some time ago, that break kernel ABI. OTOH we may be able
to merge the library with KVM backend if we really need it there.

Discussed with: rwatson

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC 196399: fstat: fix fsid comparison when executed on systems with 64-bit long

This affects only fstat on zfs and devfs, only on 64-bit systems
and only when fsid is greater than 2^31 - 1.
When fstat examines a file via stat(2) it takes uint32_t st_dev
and assigns to (signed) (64-bit) long fsid, this results in
a positive value.
When fstat examines opened files it takes int32_t f_fsid.val[0]
and assigns to (signed) (64-bit) long fsid, this results in
a negative value.
So, while initially st_dev and f_fsid.val[0] have the same bit
values they get promoted to different 64-bit values because
of the signed-vs-unsigned difference.

A fix is to use "more natural" positive numbers by introducing
intermediate unsigned cast for f_fsid.val[0].

Reviewed by: jhb, lulf
Approved by: re (kib)
MFC after: 1 week (to stable/7)

fstat: fix fsid comparison when executed on systems with 64-bit long

This affects only fstat on zfs and devfs, only on 64-bit systems
and only when fsid is greater than 2^31 - 1.
When fstat examines a file via stat(2) it takes uint32_t st_dev
and assigns to (signed) (64-bit) long fsid, this results in
a positive value.
When fstat examines opened files it takes int32_t f_fsid.val[0]
and assigns to (signed) (64-bit) long fsid, this results in
a negative value.
So, while initially st_dev and f_fsid.val[0] have the same bit
values they get promoted to different 64-bit values because
of the signed-vs-unsigned difference.

A fix is to use "more natural" positive numbers by introducing
intermediate unsigned cast for f_fsid.val[0].

Reviewed by: jhb, lulf
Approved by: re (kib)
MFC after: 1 week (to stable/7)

Don't use sys/nfs/rpcv2.h - it is part of the old kernel RPC implementation
and will be removed.

- Import the HEAD csup code which is the basis for the cvsmode work.

Integrate the new MPSAFE TTY layer to the FreeBSD operating system.

The last half year I've been working on a replacement TTY layer for the
FreeBSD kernel. The new TTY layer was designed to improve the following:

- Improved driver model:

The old TTY layer has a driver model that is not abstract enough to
make it friendly to use. A good example is the output path, where the
device drivers directly access the output buffers. This means that an
in-kernel PPP implementation must always convert network buffers into
TTY buffers.

If a PPP implementation would be built on top of the new TTY layer
(still needs a hooks layer, though), it would allow the PPP
implementation to directly hand the data to the TTY driver.

- Improved hotplugging:

With the old TTY layer, it isn't entirely safe to destroy TTY's from
the system. This implementation has a two-step destructing design,
where the driver first abandons the TTY. After all threads have left
the TTY, the TTY layer calls a routine in the driver, which can be
used to free resources (unit numbers, etc).

The pts(4) driver also implements this feature, which means
posix_openpt() will now return PTY's that are created on the fly.

- Improved performance:

One of the major improvements is the per-TTY mutex, which is expected
to improve scalability when compared to the old Giant locking.
Another change is the unbuffered copying to userspace, which is both
used on TTY device nodes and PTY masters.

Upgrading should be quite straightforward. Unlike previous versions,
existing kernel configuration files do not need to be changed, except
when they reference device drivers that are listed in UPDATING.

Obtained from: //depot/projects/mpsafetty/...
Approved by: philip (ex-mentor)
Discussed: on the lists, at BSDCan, at the DevSummit
Sponsored by: Snow B.V., the Netherlands
dcons(4) fixed by: kan

Struct cdev is always the member of the struct cdev_priv. When devfs
needed to promote cdev to cdev_priv, the si_priv pointer was followed.

Use member2struct() to calculate address of the wrapping cdev_priv.
Rename si_priv to __si_reserved.

Tested by: pho
Reviewed by: ed
MFC after: 2 weeks

Fix build of fstat after minor() changes.

Even though I ran a `make universe' to see whether the changes to the
device minor number macro's broke the build, I was not expecting `make
universe' to silently continue if build errors occured, thus causing me
to overlook the build error.

Approved by: philip (mentor)
Pointyhat to: me

Use _WANT_FILE to make struct file visible from userland. This is
similar to _WANT_UCRED and _WANT_PRISON and seems to be much nicer than
defining _KERNEL.
It is also needed for my sys/refcount.h change going in soon.

Only output details about the current working directory of a process if
the vnode pointer is not NULL. This avoids spurious warnings in fstat -v
output for kernel processes.

MFC after: 1 week
PR: amd64/123456
Submitted by: KOIE Hidetaka | hide koie.org

- Add ZFS-support to fstat(1). This allows ZFS-filsystems to be including in
the open file-listing. It is added as a separate source file, so it can
respect WITH_/WITHOUT_CDDL as compile-flags.
- The warnlevel of the Makefile was decreased to quell solaris #pragma
warnings.
- Expect that fstat(1) doesn't work with kernel compiled with
DEBUG_VFS_LOCKS/DEBUG_LOCKS for now.

Approved by: pjd (mentor)

KERN_PROC_ALL produces a kinfo_proc for each thread in a process, which
caused fstat to produce duplicated output for threaded processes. Instead
use KERN_PROC_PROC to get just one kinfo_proc per process.

MFC After: 2 weeks

Add marker vnodes to ensure that all vnodes associated with the mount point are
iterated over when using MNT_VNODE_FOREACH.

Reviewed by: truckman

Un-break processing of device major/minor values with fstat -n. We do
this by accessing the cdev_priv element of the cdev structure. Looking
forward we need a better way to handle this, as this structure shouldn't
be frobbed by userspace.

Submitted by: Doug Steinwand
PR: bin/88203
MFC after: 1 week
Discussed with: phk

Introduce kdevtoname, which when given the kernel address of a
cdev structure, returns the device name associated with it through
the __si_namebuf member. This un-breaks the processing of devices.

This is a RELENG_6 candidate.

Reviewed by: phk

Sync program's usage() with manpage's SYNOPSIS.

Dike out unwarranted si_udev fondling.

Report jail directory if set.

Sanity-check fd_lastfile.

PR: 62699
Patch by: "Mark W. Krentel" <krentel@dreamscape.com>

Get the vnode from file.f_vnode instead of file.f_data.

Nowadays, f_data points to the vnode only if the underlying filesystem
doesn't use it for other purposes (devfs uses it to store the cdev,
for example).

Found by: csjp
Reviewed by: csjp
Approved by: phk, wes, grehan (mentor)
MFC after: 1 week

filedesc0 is an internal detail of the kernel, don't look at it.

Slide pipe.h include after the _KERNEL define in preparation for disallowing
non-_KERNEL inclusions of pipe.h

1) ANSIfy.
2) Use %p to print a pointer.
3) Use longs for fileids and ino to avoid comparing signed and unsigned.
4) Make the KVM_READ macro a little more cranky.
5) Set WARNS while I'm here.

Second half of the dev_t cleanup.

The big lines are:
NODEV -> NULL
NOUDEV -> NODEV
udev_t -> dev_t
udev2dev() -> findcdev()

Various minor adjustments including handling of userland access to kernel
space struct cdev etc.

Teach fstat(1) about new location for socket state flags relating to
socket buffer state.

Submitted by: rik
Reminded by: le

fix this code properly. msdosfs can't have device nodes on them.
comment how to read device nodes from ufs (if an adventurous soul wants
to fix it!).

Reviewed by: bde

o Fix rev. 1.41, print a header. -STABLE is OK.

PR: bin/53585
Submitted by: Alexey Dokuchaev <danfe@regency.nsu.ru>

correct spelling of struct cdev * from dev_t which is a 32bit type and
isn't very useful for passing pointers on LP64 systems.

device names on sparc64 and alpha should now work.

Bow to the whining masses and change a union back into void *. Retain
removal of unnecessary casts and throw in some minor cleanups to see if
anyone complains, just for the hell of it.

Change struct file f_data to un_data, a union of the correct struct
pointer types, and remove a huge number of casts from code using it.

Change struct xfile xf_data to xun_data (ABI is still compatible).

If we need to add a #define for f_data and xf_data we can, but I don't
think it will be necessary. There are no operational changes in this
commit.

Rename struct specinfo to the more appropriate struct cdev.

Agreed on: jake, rwatson, jhb

Userland changes to go with vnode->v_tag change.

Unbreak building of fstat following version 1.23 of
src/sys/ufs/ufs/quota.h by including mount.h before ufs/quota.h.

Consistently use FBSDID

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

Factor out some code in preparation for un-kmeming fstat(1).

Sponsored by: DARPA, NAI Labs

Fix constness warnings.
Remove register keyword.
Don't initialise "badtype" in declaration - it was initialised below anyway.
Remove prototype for strcpy.

remove __P

Add support for devfs. In other words, for -CURRENT, make work at all
again for normal device nodes.

Anding the fsid with 0xffff was causing aliasing problems.

PR: 17405, 16320
Submitted by: Mark W. Krentel <krentel@dreamscape.com>
Submitted by: Peter Edwards <peter.edwards@ireland.com>
MFC after: 2 weeks

Compensate for "Compensate for header dethreading" by backing it out.

Userland part of nfs client/server split and cleanup.

SECURITY: Drop `setgid kmem' bit as early as possible.

Fix 64 bit issues so that sockstat && fstat work correctly on alpha.

PR: 29231
Submitted by: pherman@frenchfries.net
MFC after: 2 weeks

Teach fstat(1) about FIFO's - it's OK to display them as regular files.
While I'm here, fix two second-level indents to be four spaces.

Reviewed by: dd, -audit

Remove MFS

define _KERNEL before including <sys/conf.h>

Compensate for header dethreading.

Revert consequences of changes to mount.h, part 2.

Requested by: bde

Include correct header files, in preparation for fixing sys/mount.h.

Suggested-by: phk

Fix `fstat -m' (show memory-mapped files), which was broken by
revision 1.25. When evaluating the termination condition for the
iteration over all map entries, we must take care to use the kernel
versions of all pointers. The code was comparing a kernel pointer
to a pointer within a local variable, so the loop never terminated.

Change the proc information returned from the kernel so that it
no longer contains kernel specific data structures, but rather
only scalar values and structures that are already part of the
kernel/user interface, specifically rusage and rtprio. It no
longer contains proc, session, pcred, ucred, procsig, vmspace,
pstats, mtx, sigiolst, klist, callout, pasleep, or mdproc. If
any of these changed in size, ps, w, fstat, gcore, systat, and
top would all stop working. The new structure has over 200 bytes
of unassigned space for future values to be added, yet is nearly
100 bytes smaller per entry than the structure that it replaced.

remove struct mount from useland visibility

Add a new options: -m enables searching for memory-mapped files.

It is not default because it's an expensive option by nature, making the
search take 2-3 times as long.

PR: 17555
Submitted by: Ian Dowse <iedowse@maths.tcd.ie>

Support more filesystems in fstat(1): now you can use fstat(1) to
find out if files on msdosfs and cd9660 filestores are open.
There was also a movement of some common things to a header, a
small cleanup.

PR: bin/16364 bin/7043
Submitted by: Peter Edwards <peter.edwards@openet-telecom.com>

Fixed a little bit strange INET6 output.

Specified by: Ben Smithurst <ben@scientia.demon.co.uk>
Approved by: jkh

Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL"
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot). This is consistant with the other
BSD's who made this change quite some time ago. More commits to come.

Make fstat work with file arguments again after being broken by dev_t
changes. Thanks, Mr. Edwards!

Submitted by: Peter Edwards <peter.edwards@ireland.com>

$Id$ -> $FreeBSD$

o use getprotobynumber(3) instead of hardcoded protocol names.
Divert(4) sockets will now appear as "internet raw divert"
instead of "internet raw 254".

o -Wall tweak.

Reviewed by: bde

Fixed printf format errors.

Include correct header to get definition of struct sockaddr_un.

Fix argument string for getopt.

Submitted by: dillon@best.net

Display file size as an unsigned long.

Typo.

Use err(3).

Removed `#define KERNEL'. This was a hack-around for nfs.h being broken
in the Lite2 merge to not export some nfs constants. It started causing
warnings when I added a kernel-only #define for DIRBLKSIZ.

Removed `#define NFS'. This was an old, bad interface for telling
<sys/mount.h> to export nfs stuff.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Merge from Lite2

Import some parts of CSRG 4.4BSD-Lite2 usr.bin sources to fix tree build.

Include <fcntl.h> so that this doesn't depend on the KERNEL version
of <sys/file.h> including <sys/fcntl.h>. Only the !KERNEL version
of <sys/file.h> will do that when I unspam the kernel headers.

Locate the text inode, closes PR#1070

Quick attempt to decode DTYPE_PIPE files and print something meaningful
for them. Otherwise, it does strange things when it hits one..

Changes to support version 3 of the NFS protocol.
The version 2 support has been tested (client+server) against FreeBSD-2.0,
IRIX 5.3 and FreeBSD-current (using a loopback mount). The version 2 support
is stable AFAIK.
The version 3 support has been tested with a loopback mount and minimally
against an IRIX 5.3 server. It needs more testing and may have problems.
I have patched amd to support the new variable length filehandles although
it will still only use version 2 of the protocol.

Before booting a kernel with these changes, nfs clients will need to at least
build and install /usr/sbin/mount_nfs. Servers will need to build and
install /usr/sbin/mountd.

NFS diskless support is untested.

Obtained from: Rick Macklem <rick@snowhite.cis.uoguelph.ca>

Remove trailing whitespace.

Added #include <sys/queue.h>

BSD 4.4 Lite Usr.bin Sources