MFC r294137:
Do not panic when the filesystem which carries the audit files, is unmounted
and audit is active. Do not write to the suspended fs.

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

IFp4 @208383:

Currently when we discover that trail file is greater than configured
limit we send AUDIT_TRIGGER_ROTATE_KERNEL trigger to the auditd daemon
once. If for some reason auditd didn't rotate trail file it will never
be rotated.

Change it by sending the trigger when trail file size grows by the
configured limit. For example if the limit is 1MB, we will send trigger
on 1MB, 2MB, 3MB, etc.

This is also needed for the auditd change that will be committed soon
where auditd may ignore the trigger - it might be ignored if kernel
requests the trail file to be rotated too quickly (often than once a second)
which would result in overwriting previous trail file.

Sponsored by: FreeBSD Foundation (auditdistd)
MFC after: 2 weeks

IFp4 @208382:

Currently on each record write we call VFS_STATFS() to get available space
on the file system as well as VOP_GETATTR() to get trail file size.

We can assume that trail file is only updated by the audit worker, so instead
of asking for file size on every write, get file size on trail switch only
(it should be zero, but it's not expensive) and use global variable audit_size
protected by the audit worker lock to keep track of trail file's size.

This eliminates VOP_GETATTR() call for every write. VFS_STATFS() is satisfied
from in-memory data (mount->mnt_stat), so shouldn't be expensive.

Sponsored by: FreeBSD Foundation (auditdistd)
MFC after: 2 weeks

IFp4 @208381:

For VOP_GETATTR() we just need vnode to be shared-locked.

Sponsored by: FreeBSD Foundation (auditdistd)
MFC after: 2 weeks

Remove the support for using non-mpsafe filesystem modules.

In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by: attilio
Tested by: pho

Remove the thread argument from the FSD (File-System Dependent) parts of
the VFS. Now all the VFS_* functions and relating parts don't want the
context as long as it always refers to curthread.

In some points, in particular when dealing with VOPs and functions living
in the same namespace (eg. vflush) which still need to be converted,
pass curthread explicitly in order to retain the old behaviour.
Such loose ends will be fixed ASAP.

While here fix a bug: now, UFS_EXTATTR can be compiled alone without the
UFS_EXTATTR_AUTOSTART option.

VFS KPI is heavilly changed by this commit so thirdy parts modules needs
to be recompiled. Bump __FreeBSD_version in order to signal such
situation.

Wrap sx locking of the audit worker sleep lock in macros, update comments.

MFC after: 2 months
Sponsored by: Apple, Inc.

Decontextualize the couplet VOP_GETATTR / VOP_SETATTR as the passed thread
was always curthread and totally unuseful.

Tested by: Giovanni Trematerra <giovanni dot trematerra at gmail dot com>

Further synchronization of copyrights, licenses, white space, etc from
Apple and from the OpenBSM vendor tree.

Obtained from: Apple Inc., TrustedBSD Project
MFC after: 3 days

Generally avoid <space><tab> as a white space anomoly.

Obtained from: Apple Inc.
MFC after: 3 days

In preparation to sync Apple and FreeBSD versions of security audit,
pick up the Apple Computer -> Apple change in their copyright and
license templates.

Obtained from: Apple Inc.
MFC after: 3 days

When the file-system containing the audit log file is running low on
disk space a warning is printed. Make this warning a bit more
informative.

Approved by: rwatson

Don't use LK_DRAIN before calling VOP_FSYNC() in the two further
panic cases for audit trail failure -- this doesn't contribute
anything, and might arguably be wrong.

MFC after: 1 week
Requested by: attilio

Don't use LK_DRAIN before calling VOP_FSYNC() in the panic case for
audit trail failure -- this doesn't contribute anything, and might
arguably be wrong.

MFC after: 1 week
Requested by: attilio

Use __FBSDID() for $FreeBSD$ IDs in the audit code.

MFC after: 3 days

Remove XXX to remind me to check the free space calculation, which to my
eyes appears right following a check.

MFC after: 3 days

Rename globally exposed symbol send_trigger() to audit_send_trigger().

MFC after: 1 month

Replace somewhat awkward audit trail rotation scheme, which involved the
global audit mutex and condition variables, with an sx lock which protects
the trail vnode and credential while in use, and is acquired by the system
call code when rotating the trail. Previously, a "message" would be sent
to the kernel audit worker, which did the rotation, but the new code is
simpler and (hopefully) less error-prone.

Obtained from: TrustedBSD Project
MFC after: 1 month

VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used in
conjuction with 'thread' argument passing which is always curthread.
Remove the unuseful extra-argument and pass explicitly curthread to lower
layer functions, when necessary.

KPI results broken by this change, which should affect several ports, so
version bumping and manpage update will be further committed.

Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>

vn_lock() is currently only used with the 'curthread' passed as argument.
Remove this argument and pass curthread directly to underlying
VOP_LOCK1() VFS method. This modify makes the code cleaner and in
particular remove an annoying dependence helping next lockmgr() cleanup.
KPI results, obviously, changed.

Manpage and FreeBSD_version will be updated through further commits.

As a side note, would be valuable to say that next commits will address
a similar cleanup about VFS methods, in particular vop_lock1 and
vop_unlock.

Tested by: Diego Sardina <siarodx at gmail dot com>,
Andrea Di Pasquale <whyx dot it at gmail dot com>

Rename the kthread_xxx (e.g. kthread_create()) calls
to kproc_xxx as they actually make whole processes.
Thos makes way for us to add REAL kthread_create() and friends
that actually make theads. it turns out that most of these
calls actually end up being moved back to the thread version
when it's added. but we need to make this cosmetic change first.

I'd LOVE to do this rename in 7.0 so that we can eventually MFC the
new kthread_xxx() calls.

Clean up audit comments--formatting, spelling, etc.

Remove AUDIT_PRINTF() debugging statements and definition; clean up or
remove associated comments.

Slip audit_file_rotate_wait assignment in audit_rotate_vnode() before
the drop of the global audit mutex.

Obtained from: TrustedBSD Project

Short name of kernel audit worker thread from "audit_worker" to "audit".

MFC after: 3 days

Update a number of comments:

- Replace XXX with Note: in several cases where observations are made about
future functionality rather than problems or bugs.

- Remove an XXX comment about byte order and au_to_ip() -- IP headers must
be submitted in network byte order. Add a comment to this effect.

- Mention that we don't implement select/poll for /dev/audit.

Obtained from: TrustedBSD Project

Rework the way errors are handled with respect to how audit records are
written to the audit trail file:

- audit_record_write() now returns void, and all file system specific
error handling occurs inside this function. This pushes error handling
complexity out of the record demux routine that hands off to both the
trail and audit pipes, and makes trail behavior more consistent with
pipes as a record destination.

- Rate limit kernel printfs associated with running low on space. Rate
limit audit triggers for low space. Rate limit printfs for fail stop
events. Rate limit audit worker write error printfs.

- Document in detail the types of limits and space checks we perform, and
combine common cases.

This improves the audit subsystems tolerance to low space conditions by
avoiding toasting the console with printfs are waking up the audit daemon
continuously.

MFC after: 3 days
Obtained from: TrustedBSD Project

Merge OpenBSM 1.0 alpha 11 changes into src/sys/bsm and src/sys/security;
primarily, add new event identifiers and update trigger names.

Obtained from: TrustedBSD Project

Correct a slight regression which was introduced with the implementation of
audit pipes. If the kernel record was not selected for the trail or the pipe,
any user supplied record attached to it would be tossed away, resulting in
otherwise selected events being lost.

- Introduce two new masks: AR_PRESELECT_USER_TRAIL AR_PRESELECT_USER_PIPE,
currently we have AR_PRESELECT_TRAIL and AR_PRESELECT_PIPE, which tells
the audit worker that we are interested in the kernel record, with
the additional masks we can determine if either the pipe or trail is
interested in seeing the kernel or user record.

- In audit(2), we unconditionally set the AR_PRESELECT_USER_TRAIL and
AR_PRESELECT_USER_PIPE masks under the assumption that userspace has
done the preselection [1].

Currently, there is work being done that allows the kernel to parse and
preselect user supplied records, so in the future preselection could occur
in either layer. But there is still a few details to work out here.

[1] At some point we need to teach au_preselect(3) about the interests of
all the individual audit pipes.

This is a RELENG_6 candidate.

Reviewed by: rwatson
Obtained from: TrustedBSD Project
MFC after: 1 week

Extract pointer value for mnt_stat from vp after the NULL check, not
before.

Coverity ID: 134394
Found with: Coverity Prevent (tm)

Introduce support for per-audit pipe preselection independent from the
global audit trail configuration. This allows applications consuming
audit trails to specify parameters for which audit records are of
interest, including selecting records not required by the global trail.
Allowing application interest specification without changing the global
configuration allows intrusion detection systems to run without
interfering with global auditing or each other (if multiple are
present). To implement this:

- Kernel audit records now carry a flag to indicate whether they have
been selected by the global trail or by the audit pipe subsystem,
set during record commit, so that this information is available
after BSM conversion when delivering the BSM to the trail and audit
pipes in the audit worker thread asynchronously. Preselection by
either record target will cause the record to be kept.

- Similar changes to preselection when the audit record is created
when the system call is entering: consult both the global trail and
pipes.

- au_preselect() now accepts the class in order to avoid repeatedly
looking up the mask for each preselection test.

- Define a series of ioctls that allow applications to specify whether
they want to track the global trail, or program their own
preselection parameters: they may specify their own flags and naflags
masks, similar to the global masks of the same name, as well as a set
of per-auid masks. They also set a per-pipe mode specifying whether
they track the global trail, or user their own -- the door is left
open for future additional modes. A new ioctl is defined to allow a
user process to flush the current audit pipe queue, which can be used
after reprogramming pre-selection to make sure that only records of
interest are received in future reads.

- Audit pipe data structures are extended to hold the additional fields
necessary to support preselection. By default, audit pipes track the
global trail, so "praudit /dev/auditpipe" will track the global audit
trail even though praudit doesn't program the audit pipe selection
model.

- Comment about the complexities of potentially adding partial read
support to audit pipes.

By using a set of ioctls, applications can select which records are of
interest, and toggle the preselection mode.

Obtained from: TrustedBSD Project

No longer unconditionally drain the audit record queue if there is
not an active audit trail: instead, continue to iterate through
each record in case an audit pipe is interested.

Obtained from: TrustedBSD Project

Pull BSM conversion logic out of audit_record_write(), as well as
knowledge of user vs. kernel audit records into
audit_worker_process_record(). This largely confines vnode
knowledge to audit_record_write(), but avoids that logic knowing
about BSM as opposed to byte streams. This will allow us to
improve our ability to support real-time audit stream processing
by audit pipe consumers while auditing is disabled, but this
support is not yet complete.

Obtained from: TrustedBSD Project

Assert audit mtx in audit_worker_drain().

Break out logic to call audit_record_write() and handle error
conditions into audit_worker_process_record(). This will be the
future home of some logic now present in audit_record_write()
also.

Obtained from: TrustedBSD Project

Use struct kaudit_queue instead of a hand-crafted queue type for
audit records in the audit_worker thread.

Obtained from: TrustedBSD Project

Rename audit_cv to audit_worker_cv, as it wakes up the audit
worker.

Rename audit_commit_cv to audit_watermark_cv, since it is there to
wake up threads waiting on hitting the low watermark. Describe
properly in comment.

Obtained from: TrustedBSD Project

Merge Perforce change 93581 from TrustedBSD audit3 branch:

Mega-style patch.

Obtained from: TrustedBSD Project

Merge Perforce changes 93512, 93514, 93515 from TrustedBSD audit3
branch:

Integrate audit.c to audit_worker.c, so as to migrate the worker
thread implementation to its own .c file.

Populate audit_worker.c using parts now removed from audit.c:

- Move audit rotation global variables.
- Move audit_record_write(), audit_worker_rotate(),
audit_worker_drain(), audit_worker(), audit_rotate_vnode().
- Create audit_worker_init() from relevant parts of audit_init(),
which now calls this routine.
- Recreate audit_free(), which wraps uma_zfree() so that
audit_record_zone can be static to audit.c.
- Unstaticize various types and variables relating to the audit
record queue so that audit_worker can get to them. We may want
to wrap these in accessor methods at some point.
- Move AUDIT_PRINTF() to audit_private.h.

Addition of audit_worker.c to kernel configuration, missed in
earlier submit.

Obtained from: TrustedBSD Project