Add interface NVME to devstat

This allows to list only NVMe devices in systat, iostat, vmstat, etc.
Previously those were counted as OTHER.

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

Fix asymmetry in devstat(9) calls by GEOM.

Before this GEOM passed bio pointer to transaction start, but not end.
It was irrelevant until devstat(9) got DTrace hooks, that appeared to
provide bio pointer on I/O completion, but not on submission.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

devstat(9): Constify function parameters that can be const

No functional change.

When attempting to document the changed argument types in devstat.9, I
discovered the 20 year old manual page severely mismatched reality even
prior to my simple change. So I took a first cut pass cleaning that up to
match reality. I'm sure I've missed some things; the goal was just to leave
it better than when I started.

Sponsored by: Dell EMC Isilon

sys/sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

MFprojects/camlock r254905:
Introduce new function devstat_end_transaction_bio_bt(), adding new argument
to specify present time. Use this function to move binuptime() out of lock,
substantially reducing lock congestion when slow timecounter is used.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

- Import the HEAD csup code which is the basis for the cvsmode work.

/* -> /*- for license, minor formatting changes

Add an XXX comment with a TODO item for next time we run a revision
on this API.

Make devstat_new_entry() take a const void * rather than const char *
argument, GEOM nodes are not identified by ascii string.

Add a #define for the device name of the mmap device for devstat.
Constify the geom identification pointer.

Run a revision of the devstat interface:

Kernel:

Change statistics to use the *uptime() timescale (ie: relative to
boottime) rather than the UTC aligned timescale. This makes the
device statistics code oblivious to clock steps.

Change timestamps to bintime format, they are cheaper.

Remove the "busy_count", and replace it with two counter fields:
"start_count" and "end_count", which are updated in the down and
up paths respectively. This removes the locking constraint on
devstat.

Add a timestamp argument to devstat_start_transaction(), this will
normally be a timestamp set by the *_bio() function in bp->bio_t0.
Use this field to calculate duration of I/O operations.

Add two timestamp arguments to devstat_end_transaction(), one is
the current time, a NULL pointer means "take timestamp yourself",
the other is the timestamp of when this transaction started (see
above).

Change calculation of busy_time to operate on "the salami principle":
Only when we are idle, which we can determine by the start+end
counts being identical, do we update the "busy_from" field in the
down path. In the up path we accumulate the timeslice in busy_time
and update busy_from.

Change the byte_* and num_* fields into two arrays: bytes[] and
operations[].

Userland:

Change the misleading "busy_time" name to be called "snap_time" and
make the time long double since that is what most users need anyway,
fill it using clock_gettime(CLOCK_MONOTONIC) to put it on the same
timescale as the kernel fields.

Change devstat_compute_etime() to operate on struct bintime.

Remove the version 2 legacy interface: the change to bintime makes
compatibility far too expensive.

Fix a bug in systat's "vm" page where boot relative busy times would
be bogus.

Bump __FreeBSD_version to 500107

Review & Collaboration by: ken

Add a devstat_start_transaction_bio() to match the
devstat_end_transaction_bio() we already have.

For now it just calls devstat_start_transaction(), but that will change
shortly.

Retire devstat_add_entry() as a public function and bump __FreeBSD_version
to mark this act.

Augment the devstat structure with 3 new fields and bump the DEVSTAT_VERSION.

Two fields are sequence numbers for integrity check when we switch devstat
to use mmap to export data rather than sysctl, the last field is to mark
this as an allocated devstat entry.

Un-inline the non-trivial "trivial" bio* functions.
Untangle devstat_end_transaction_bio()

Export the head structure for the device statistics STAILQ in
sys/devicestat.h, so that the queue can be walked in crashdumps using
libkvm.

Actually biofinish(struct bio *, struct devstat *, int error) is more general
than the bioerror().

Most of this patch is generated by scripts.

Back out the previous change to the queue(3) interface.
It was not discussed and should probably not happen.

Requested by: msmith and others

Change the way that the queue(3) structures are declared; don't assume that
the type argument to *_HEAD and *_ENTRY is a struct.

Suggested by: phk
Reviewed by: phk
Approved by: mdodd

Name the `struct bio *' parameter of devstat_end_transaction_bio().

Removed the last vestige of devstat_end_transaction_buf().

Remove devstat_end_transaction_buf() everybody uses
devstat_end_transaction_bio() now.

Silence some warnings.

Clone bio versions of certain bits of infrastructure:
devstat_end_transaction_bio()
bioq_* versions of bufq_* incl bioqdisksort()
the corresponding "buf" versions will disappear when no longer used.

Move b_offset, b_data and b_bcount to struct bio.

Add BIO_FORMAT as a hack for fd.c etc.

We are now largely ready to start converting drivers to use struct
bio instead of struct buf.

Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL"
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot). This is consistant with the other
BSD's who made this change quite some time ago. More commits to come.

Revamp the devstat priority system. All disks now have the same priority.
The same goes for CD drivers and tape drivers. In systems with mixed IDE
and SCSI, devices in the same priority class will be sorted in attach
order.

Also, the 'CCD' priority is now the 'ARRAY' priority, and a number of
drivers have been modified to use that priority.

This includes the necessary changes to all drivers, except the ATA drivers.
Soren will modify those separately.

This does not include and does not require any change in the devstat
version number, since no known userland applications use the priority
enumerations.

Reviewed by: msmith, sos, phk, jlemon, mjacob, bde

Bump version number.

Pointed out by: ken

Count "free" operations in their own new tranaction type.

WARNING: libdevstat, iostat, vmstat, systat etc etc will need a recompile.

Add devstat_end_transaction_buf() which pulls all the vital data out
of a struct buf which is ready for biodone().

$Id$ -> $FreeBSD$

Add a prioritization field to the devstat_add_entry() call so that
peripheral drivers can determine where in the devstat(9) list they are
inserted.

This requires recompilation of libdevstat, systat, vmstat, rpc.rstatd, and
any ports that depend on the devstat code, since the size of the devstat
structure has changed. The devstat version number has been incremented as
well to reflect the change.

This sorts devices in the devstat list in "more interesting" to "less
interesting" order. So, for instance, da devices are now more important
than floppy drives, and so will appear before floppy drives in the default
output from systat, iostat, vmstat, etc.

The order of devices is, for now, kept in a central table in devicestat.h.
If individual drivers were able to make a meaningful decision on what
priority they should be at attach time, we could consider splitting the
priority information out into the various drivers. For now, though, they
have no way of knowing that, so it's easier to put them in an easy to find
table.

Also, move the checkversion() call in vmstat(8) to a more logical place.

Thanks to Bruce and David O'Brien for suggestions, for reviewing this, and
for putting up with the long time it has taken me to commit it. Bruce did
object somewhat to the central priority table (he would rather the
priorities be distributed in each driver), so his objection is duly noted
here.

Reviewed by: bde, obrien

My changes to the new device interface:

- Interface wth the new resource manager.
- Allow for multiple drivers implementing a single devclass.
- Remove ordering dependencies between header files.
- Style cleanup.
- Add DEVICE_SUSPEND and DEVICE_RESUME methods.
- Move to a single-phase interrupt setup scheme.

Kernel builds on the Alpha are brken until Doug gets a chance to incorporate
these changes on that side.

Agreed to in principle by: dfr

Change the devstat generation number from an int to a long. The int-sized
generation was causing unaligned access faults on the Alpha.

I have incremented the devstat version number, since this is an interface
change. You'll need to recompile libdevstat, systat, iostat, vmstat and
rpc.rstatd along with your kernel.

Partially Submitted by: Andrew Gallatin <gallatin@cs.duke.edu>

New Kernel device statistics code.

Submitted by: "Kenneth D. Merry" <ken@plutotech.com>