sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

filemon: Reject FILEMON_SET_FD commands when the fd is a kqueue

When FILEMON_SET_FD is used, the filemon handle effectively wraps the
passed file. In particular, the handle may be inherited by a child
process, or transferred over a unix domain socket, so we must verify
that the backing file permits this.

Reported by: syzbot+36e6be9e02735fe66ca8@syzkaller.appspotmail.com
Reviewed by: emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34128

Fix some filemon path logging issues.

- Properly handle snprintf return value for truncation and avoid
overflowing the later write with the bogus length.
- Increase the msgbufr size to handle a rename of 2 full files.

The larger allocation causes a slight performance hit which will be mitigated
in the future. A rewrite with sbufs will likely be done as well.

Reported by: Ilja Van Sprundel <ivansprundel@ioactive.com>
MFC after: 2 weeks
Approved by: so (gtetlow)
Reviewed by: kib
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16098

Eliminate the overhead of gratuitous repeated reinitialization of cap_rights

- Add macros to allow preinitialization of cap_rights_t.

- Convert most commonly used code paths to use preinitialized cap_rights_t.
A 3.6% speedup in fstat was measured with this change.

Reported by: mjg
Reviewed by: oshogbo
Approved by: sbruno
MFC after: 1 month

Move most of the contents of opt_compat.h to opt_global.h.

opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c. A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by: kib, cem, jhb, jtl
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14941

sys/dev: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Avoid taking PROC_LOCK in syscalls if not being traced.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Cleanup COMPAT_FREEBSD32 support.

This is a NOP.

The COMPAT_IA32 was renamed in r205014 to COMPAT_FREEBSD32 and
COMPAT_ARCH32 does not seem to have existed. Also remove some
leftovers from the sysent rework in r301404. Include
freebsd32_util.h for the freebsd32_sysent prototype.

X-MFC-With: r301404
Reported by: kib
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Rename function to be less generic.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

exec: Cease tracing if credentials will change with the new image.

This also prevents tracing to a P_INEXEC process since it could race
with other processes attaching to it in filemon_event_process_exec() due
to the filemon_get_proc() race of incrementing ref and then locking the
filemon. With the no-P_INEXEC invariant in place the p_filemon may only
be the same or NULL when trying to drop it in
filemon_event_process_exec().

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D6545

Remove unneeded return left from refactoring.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Return any log write failure encountered when closing the filemon fd.

Discussed with: sjg, markj
Reviewed by: sjg
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Follow-up r297156: Close the log in filemon_dtr rather than in the last reference.

If the tracer has decided to the close the log then it should be fully
written, not getting more entries, when close(2) returns. This was
a regression in r297156 in that it allowed a traced process to continue
a traced syscall and add more entries to the log while the tracer had
already closed its fd or exited. This was only really part of the
daemonized process case which is abnormal.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Track filemon usage via a proc.p_filemon pointer rather than its own lists.

- proc.p_filemon is added which is protected by PROC_LOCK. This improves
performance and avoids double-fork issues, taking allproc_lock
while in syscalls, and walking the process tree in syscalls. A
particular proc.p_filemon can only be changed to NULL or another
filemon, or the filemon inherited, while the filemon->lock is held.
- Filemon are reference counted. On the last reference the log will be closed.
- When closing the devfs file handle, the filemon will be detached from all
processes and inheritance prevented.
- Disallow attaching to a process already being traced since filemon is
typically intended to be used on children only. This is allowed for
curproc as bmake relies on this behavior for rare cases when combining
.MAKE with .META.
- Detach any previously tracked process on ioctl(FILEMON_SET_PID).
- Handle error from devfs_set_cdevpriv() in filemon_open().
- The global filemon lock and lists are removed.
- A free list is no longer kept. Previously this list was
forever-expanding and never garbage cleaned.
- No longer loses track of double-forks. If the process holding the filemon
handle closes it will close the log rather than wait on a daemonized process,
but it will log all activity until it closes its handle. The filemon
will be removed from the process and not inherited.
- A separate process count is kept only as an optimization for
forced detachment to avoid taking allproc_lock and walking the entire
process tree.
- struct filemon access is protected by sx(9) filemon->lock as it was before.
- Add more comments and KASSERTS.

MFC after: 2 weeks
Reviewed by: kib, mjg, markj (all on previous versions)
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D5520

FILEMON_SET_FD: Disallow changing the fd.

MFC after: 1 week
Suggested by: mjg
Sponsored by: EMC / Isilon Storage Division

Add missing break for r296472.

This was lost in git rebasing, though it has no functional change.

X-MFC-With: r296472
MFC after: 1 week

Require kldunload -f to unload.

Code may still be executing from the wrappers at unload time and thus is
not generally safe to unload. Converting the wrappers to use
EVENTHANDLER(9) will allow this to safely drain on active threads in
hooks. More work on EVENTHANDLER(9) is needed first.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Remove filemon->lock wrappers.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Remove old compatibility checks.

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

filemon: Track the process pointer rather than a pid.

The process is not held since the process_exit hook is called after the
exithold. There is no need to hold the process since the hook will
always see it exiting via the process_exit event.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

filemon_open: Don't record a process to trace here.

Only ioctl(FILEMON_SET_PID) should be setting the process-to-be-traced.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

filemon_dtr: Lock the associated filemon handle before writing to it.

Reported by: mjg
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

filemon_comment has nothing to do with wrappers so move it out of filemon_wrapper.c.

It only prints the header from filemon_ioctl. Keep the name though to stay
closer to other implementations.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

filemon_ioctl: Lock the associated filemon handle before writing to it.

Reported by: mjg
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

filemon_ioctl: Handle error from devfs_get_cdevpriv(9).

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Drop any previous fd when setting a new one.

Reported by: mjg
Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks

Remove unneeded mutex.h include, missed in r287155.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Remove unneeded NULL as this is initialized with M_ZERO.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Fix filemon locking races.

Convert filemon_lock and struct filemon* lock to sx(9), rather than a
self-rolled reader-writer lock, and hold it for the entire time needed.

At least filemon_lock_write() was not checking for active readers when
it would successfully return with the write lock "held". This led to
a race with reading entries from filemon_inuse as they were removed. This
could be seen with QUEUE_MACRO_DEBUG enabled, causing -1 to be read as an
entry rather than a valid struct filemon*.

Fixing filemon_lock_write() to check readers was insufficient to fix the
races.

sx(9) was used as the lock could be held while taking proctree_lock and sleeping
in fo_write.

Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks

Update kernel inclusions of capability.h to use capsicum.h instead; some
further refinement is required as some device drivers intended to be
portable over FreeBSD versions rely on __FreeBSD_version to decide whether
to include capability.h.

MFC after: 3 weeks

Change the cap_rights_t type from uint64_t to a structure that we can extend
in the future in a backward compatible (API and ABI) way.

The cap_rights_t represents capability rights. We used to use one bit to
represent one right, but we are running out of spare bits. Currently the new
structure provides place for 114 rights (so 50 more than the previous
cap_rights_t), but it is possible to grow the structure to hold at least 285
rights, although we can make it even larger if 285 rights won't be enough.

The structure definition looks like this:

struct cap_rights {
uint64_t cr_rights[CAP_RIGHTS_VERSION + 2];
};

The initial CAP_RIGHTS_VERSION is 0.

The top two bits in the first element of the cr_rights[] array contain total
number of elements in the array - 2. This means if those two bits are equal to
0, we have 2 array elements.

The top two bits in all remaining array elements should be 0.
The next five bits in all array elements contain array index. Only one bit is
used and bit position in this five-bits range defines array index. This means
there can be at most five array elements in the future.

To define new right the CAPRIGHT() macro must be used. The macro takes two
arguments - an array index and a bit to set, eg.

#define CAP_PDKILL CAPRIGHT(1, 0x0000000000000800ULL)

We still support aliases that combine few rights, but the rights have to belong
to the same array element, eg:

#define CAP_LOOKUP CAPRIGHT(0, 0x0000000000000400ULL)
#define CAP_FCHMOD CAPRIGHT(0, 0x0000000000002000ULL)

#define CAP_FCHMODAT (CAP_FCHMOD | CAP_LOOKUP)

There is new API to manage the new cap_rights_t structure:

cap_rights_t *cap_rights_init(cap_rights_t *rights, ...);
void cap_rights_set(cap_rights_t *rights, ...);
void cap_rights_clear(cap_rights_t *rights, ...);
bool cap_rights_is_set(const cap_rights_t *rights, ...);

bool cap_rights_is_valid(const cap_rights_t *rights);
void cap_rights_merge(cap_rights_t *dst, const cap_rights_t *src);
void cap_rights_remove(cap_rights_t *dst, const cap_rights_t *src);
bool cap_rights_contains(const cap_rights_t *big, const cap_rights_t *little);

Capability rights to the cap_rights_init(), cap_rights_set(),
cap_rights_clear() and cap_rights_is_set() functions are provided by
separating them with commas, eg:

cap_rights_t rights;

cap_rights_init(&rights, CAP_READ, CAP_WRITE, CAP_FSTAT);

There is no need to terminate the list of rights, as those functions are
actually macros that take care of the termination, eg:

#define cap_rights_set(rights, ...) \
__cap_rights_set((rights), __VA_ARGS__, 0ULL)
void __cap_rights_set(cap_rights_t *rights, ...);

Thanks to using one bit as an array index we can assert in those functions that
there are no two rights belonging to different array elements provided
together. For example this is illegal and will be detected, because CAP_LOOKUP
belongs to element 0 and CAP_PDKILL to element 1:

cap_rights_init(&rights, CAP_LOOKUP | CAP_PDKILL);

Providing several rights that belongs to the same array's element this way is
correct, but is not advised. It should only be used for aliases definition.

This commit also breaks compatibility with some existing Capsicum system calls,
but I see no other way to do that. This should be fine as Capsicum is still
experimental and this change is not going to 9.x.

Sponsored by: The FreeBSD Foundation

- Use pget(PGET_CANDEBUG | PGET_NOTWEXIT) to determine if the specified
PID is valid for monitoring in FILEMON_SET_PID ioctl.

- Set the monitored PID to -1 when the process exits.

Suggested by: jilles
Tested by: sjg
MFC after: 3 days

Add p_candebug() check to FILEMON_SET_PID ioctl.

Discussed with: sjg
MFC after: 3 days

Build fix: Only <sys/cdefs.h> should be included before __FBSDID().
<sys/param.h> needs to be included after any "opt_foo.h" headers so it
sees the same set of defined macros as other headers.

Match the options of the kernel.

A little bit easier to read.

Desupport pre-FreeBSD 7.1.

Add the 'filemon' device. 'filemon' is a kernel module that provides a device
interface for processes to record system calls of its children.

Submitted by: Juniper Networks.