alq, siftr: add panic/debugger checks to shutdown hooks

Don't try to gracefully terminate the pkt_manager thread if the
scheduler is not running.

We should not attempt to shutdown ald if RB_NOSYNC is set, and must not
if the scheduler is stopped (the function calls wakeup()).

Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42340

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

vfs: NDFREE(&nd, NDF_ONLY_PNBUF) -> NDFREE_PNBUF(&nd)

vfs: remove the unused thread argument from NDINIT*

See b4a58fbf640409a1 ("vfs: remove cn_thread")

Bump __FreeBSD_version to 1400043.

alq_open_flags(): mark local td variable as unused

It is passed to the NDINIT() macro which ignores the thread argument
for some time.

Sponsored by: The FreeBSD Foundation

kern: clean up empty lines in .c and .h files

Remove duplicated empty lines from kern/*.c

No functional changes.

vfs: drop the mostly unused flags argument from VOP_UNLOCK

Filesystems which want to use it in limited capacity can employ the
VOP_UNLOCK_FLAGS macro.

Reviewed by: kib (previous version)
Differential Revision: https://reviews.freebsd.org/D21427

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Use SI_SUB_LAST instead of SI_SUB_SMP as the "catch-all" subsystem.

Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D5515

Prevent alq from panic when the invalid alq_file path specified.

MFC after: 1 week

The fix committed in r250951 replaced the reported panic with a deadlock... gold
star for me. EVENTHANDLER_DEREGISTER() attempts to acquire the lock which is
held by the event handler framework while executing event handler functions,
leading to deadlock.

Move EVENTHANDLER_DEREGISTER() to alq_load_handler() and thus deregister the ALQ
shutdown_pre_sync handler at module unload time, which takes care of the
originally reported panic and fixes the deadlock introduced in r250951.

Reported by: Luiz Otavio O Souza
MFC after: 3 days
X-MFC with: 250951

Ensure alq's shutdown_pre_sync event handler is deregistered on module unload to
avoid a dangling pointer and eventual panic on system shutdown.

Reported by: Ali <comnetboy at gmail.com>
Tested by: Ali <comnetboy at gmail.com>
MFC after: 1 week

Remove the support for using non-mpsafe filesystem modules.

In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by: attilio
Tested by: pho

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

- Rework the underlying ALQ storage to be a circular buffer, which amongst other
things allows variable length messages to be easily supported.

- Extend KPI with alq_writen() and alq_getn() to support variable length
messages, which is enabled at ALQ creation time depending on the
arguments passed to alq_open(). Also add variants of alq_open() and
alq_post() that accept a flags argument. The KPI is still fully
backwards compatible and shouldn't require any change in ALQ consumers
unless they wish to utilise the new features.

- Introduce the ALQ_NOACTIVATE and ALQ_ORDERED flags to allow ALQ consumers
to have more control over IO scheduling and resource acquisition
respectively.

- Strengthen invariants checking.

- Document ALQ changes in ALQ(9) man page.

Sponsored by: FreeBSD Foundation
Reviewed by: gnn, jeff, rpaulo, rwatson
MFC after: 1 month

The ALQ should not be considered drained until it has been made inactive.

Sponsored by: FreeBSD Foundation
Reviewed by: dwmalone, jeff, rpaulo, rwatson (as part of a larger patch)
Approved by: kmacy (mentor)
MFC after: 1 month

According to SLEEP(9), msleep() is deprecated in favour of mtx_sleep().

Sponsored by: FreeBSD Foundation
Reviewed by: dwmalone, jeff, rpaulo, rwatson (as part of a larger patch)
Approved by: kmacy (mentor)
MFC after: 1 month

- Factor code to destroy an ALQ out of alq_close() into a private alq_destroy().

- Use the new alq_destroy() to properly handle a failure case in alq_open().

Sponsored by: FreeBSD Foundation
Reviewed by: dwmalone, jeff, rpaulo, rwatson (as part of a larger patch)
Approved by: kmacy (mentor)
MFC after: 1 month

Add support for ALQ(9) to be compiled and loaded as a kernel module.

Sponsored by: FreeBSD Foundation
Reviewed by: dwmalone, jeff, rpaulo, rwatson
Approved by: kmacy (mentor)
MFC after: 1 month

Add another flags argument to vn_open_cred. Use it to specify that some
vn_open_cred invocations shall not audit namei path.

In particular, specify VN_OPEN_NOAUDIT for dotdot lookup performed by
default implementation of vop_vptocnp, and for the open done for core
file. vn_fullpath is called from the audit code, and vn_open there need
to disable audit to avoid infinite recursion. Core file is created on
return to user mode, that, in particular, happens during syscall return.
The creation of the core file is audited by direct calls, and we do not
want to overwrite audit information for syscall.

Reported, reviewed and tested by: rwatson

Move "options MAC" from opt_mac.h to opt_global.h, as it's now in GENERIC
and used in a large number of files, but also because an increasing number
of incorrect uses of MAC calls were sneaking in due to copy-and-paste of
MAC-aware code without the associated opt_mac.h include.

Discussed with: pjd

Remove VOP_LEASE and supporting functions. This hasn't been used since
the removal of NQNFS, but was left in in case it was required for NFSv4.
Since our new NFSv4 client and server can't use it for their
requirements, GC the old mechanism, as well as other unused lease-
related code and interfaces.

Due to its impact on kernel programming and binary interfaces, this
change should not be MFC'd.

Proposed by: jeff
Reviewed by: jeff
Discussed with: rmacklem, zach loafman @ isilon

- Import the HEAD csup code which is the basis for the cvsmode work.

Use msleep_spin() instead of unlock/tsleep/lock. This was
already commited but with a wrong msleep variant and then
backed out. Note that this changes the semantic a little
as msleep_spin does not let us to specify priority after
wakeup.

Approved by: wkoszek, cognet
Approved by: kib (mentor)

In keeping with style(9)'s recommendations on macros, use a ';'
after each SYSINIT() macro invocation. This makes a number of
lightweight C parsers much happier with the FreeBSD kernel
source, including cflow's prcc and lxr.

MFC after: 1 month
Discussed with: imp, rink

VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used in
conjuction with 'thread' argument passing which is always curthread.
Remove the unuseful extra-argument and pass explicitly curthread to lower
layer functions, when necessary.

KPI results broken by this change, which should affect several ports, so
version bumping and manpage update will be further committed.

Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>

vn_lock() is currently only used with the 'curthread' passed as argument.
Remove this argument and pass curthread directly to underlying
VOP_LOCK1() VFS method. This modify makes the code cleaner and in
particular remove an annoying dependence helping next lockmgr() cleanup.
KPI results, obviously, changed.

Manpage and FreeBSD_version will be updated through further commits.

As a side note, would be valuable to say that next commits will address
a similar cleanup about VFS methods, in particular vop_lock1 and
vop_unlock.

Tested by: Diego Sardina <siarodx at gmail dot com>,
Andrea Di Pasquale <whyx dot it at gmail dot com>

Merge first in a series of TrustedBSD MAC Framework KPI changes
from Mac OS X Leopard--rationalize naming for entry points to
the following general forms:

mac_<object>_<method/action>
mac_<object>_check_<method/action>

The previous naming scheme was inconsistent and mostly
reversed from the new scheme. Also, make object types more
consistent and remove spaces from object types that contain
multiple parts ("posix_sem" -> "posixsem") to make mechanical
parsing easier. Introduce a new "netinet" object type for
certain IPv4/IPv6-related methods. Also simplify, slightly,
some entry point names.

All MAC policy modules will need to be recompiled, and modules
not updates as part of this commit will need to be modified to
conform to the new KPI.

Sponsored by: SPARTA (original patches against Mac OS X)
Obtained from: TrustedBSD Project, Apple Computer

Change the VOP_OPEN(), vn_open() vnode operation and d_fdopen() cdev operation
argument from being file descriptor index into the pointer to struct file:
part 2. Convert calls missed in the first big commit.

Noted by: rwatson
Pointy hat to: kib

Backout rev 1.17, msleep() can't be used with a spinlock.

Pointy hat to: cognet

Use msleep(9) instead of tsleep(9) surrounded by lock acquisition and
release.

Approved by: cognet (mentor)

Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h
begun with a repo-copy of mac.h to mac_framework.h. sys/mac.h now
contains the userspace and user<->kernel API and definitions, with all
in-kernel interfaces moved to mac_framework.h, which is now included
across most of the kernel instead.

This change is the first step in a larger cleanup and sweep of MAC
Framework interfaces in the kernel, and will not be MFC'd.

Obtained from: TrustedBSD Project
Sponsored by: SPARTA

- Conditionalize Giant around VFS operations for ALQ, ktrace, and
generating a coredump as the result of a signal.
- Fix a bug where we could leak a Giant lock if vn_start_write() failed
in coredump().

Reported by: jmg (2)

Grr. Backout previous change. vn_open_cred() will call NDFREE() on failure.

Don't forget to call NDFREE(9) in case of vn_open_cred() failure.

MFC after: 3 days

Modify the alq(9) alq_open() API to accept a file creation mode, rather
than defaulting the cmode argument to vn_open() to 0. Supply a default
argument of ALQ_DEFAULT_CMODE (0600) in current callers.

Discussed with/pointed out by: hmp
Reveiwed by: jeff, hmp
MFC after: 3 days

/* -> /*- for copyright notices, minor format tweaks as necessary

Allow MAC policies to block/revoke kern_alq write access to a file.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories
Reviewed by: jeff

Add fdidx argument to vn_open() and vn_open_cred() and pass -1 throughout.

Add an explicit credential argument to alq_open() to allow the caller to
specify what credential to use when authorizing vn_open() and later
write operations, rather than curthread->td_ucred.

When writing KTR traces to an ALQ, specify the credential of the thread
generating the sysctl request.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Use __FBSDID().

- Reset the free ent to NULL if we have consumed the last free entry. This
fixes a problem where we would overwrite old data if we ran out of free
entries.

Submitted by: sam
Approved by: re (scottl)

Back out M_* changes, per decision of the TRB.

Approved by: trb

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

- Export the alq daemon thread pointer.
- Don't log ktr events from the alq daemon.

- Hold the credential of the caller and use it in all subsequent vn ops.
- Get rid of the ill conceived aq_td field.

Suggested by: rwatson

- Add an asynchronous fixed length record logging mechanism called
ALQ (Asynch. Logging Queues). ALQ supports many seperate queues with
different record and buffer sizes. It opens and logs to any vnode so
it can be used with character devices as well as regular files.

Reviewed in part by: phk, jake, markm