sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

GEOM: Introduce partial confxml API

Traditionally the GEOM's primary channel of information from kernel to
user-space was confxml, fetched by libgeom through kern.geom.confxml
sysctl. It is convenient and informative, representing full state of
GEOM in a single XML document. But problems start to arise on systems
with hundreds of disks, where the full confxml size reaches many
megabytes, taking significant time to first write it and then parse.

This patch introduces alternative solution, allowing to fetch much
smaller XML document, subset of the full confxml, limited to 64KB and
representing only one specified geom and optionally its parents. It
uses existing GEOM control interface, extended with new "getxml" verb.
In case of any error, such as the buffer overflow, it just transparently
falls back to traditional full confxml. This patch uses the new API in
user-space GEOM tools where it is possible.

Reviewed by: imp
MFC after: 2 month
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D34529

geom: clean up empty lines in .c and .h files

Reimplement aliases in geom

The alias needs to be part of the provider instead of the geom to work
properly. To bind the DEV geom, we need to look at the provider's names and
aliases and create the dev entries from there. If this lives in the GEOM, then
it won't propigate down the tree properly. Remove it from geom, add it provider.

Update geli, gmountver, gnop, gpart, and guzip to use it, which handles the bulk
of the uses in FreeBSD. I think this is all the providers that create a new name
based on their parent's name.

Wrap g_trace() into a macro to avoid unneeded calls.

In most cases with debug disabled this function does nothing, but argument
passing and the call still cost measurable time due to cache misses, etc.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Use sbuf_cat() in GEOM confxml generation.

When it comes to megabytes of text, difference between sbuf_printf() and
sbuf_cat() becomes substantial.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Extend stripeoffset and stripesize of GEOMs from u_int to off_t

GEOM's stripeoffset overflows at 4 gigabyte margin (2^32)
because of its u_int type. This leads to incorrect data in the output
generated by "sysctl kern.geom.confxml" command, "graid list" etc.
when GEOM array has volumes larger than 4G, for example.

This change does not affect ABI but changes KBI. No MFC planned.

Differential Revision: https://reviews.freebsd.org/D13426

sys/geom: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Add aliasing concept to geom.

Add an alias name list to geoms. Use them in geom_dev to create
aliases. Previously, geom_dev would create an device node for the name
of the geom. Now, additional nodes are created pointing back to the
primary node with make_dev_alias_p. Aliases must be in place on the
geom before any tasting occurs.

Differential Revision: https://reviews.freebsd.org/D11873

Report withered providers as such alike to GEOMs.

MFC after: 2 weeks

Add provider's sectorsize and stripesize to confdot output.

Submitted by: rpokala at panasas.com

Escape special XML chars, returned by some devices, confusing XML parsers.

MFC after: 1 month

Remove an extra semicolon from the DOT language output.

PR: kern/178540
Submitted by: Trond Endrestol
MFC after: 1 week

Introduce a symbol for the GEOM class name instead of using the ad-hoc string
constant.

Do not hide stripeoffset from libgeom(3), it may be useful even when
stripesize is zero.

MFC after: 1 week

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFC r205385:

Escape characters unsafe for XML output in GEOM class, instance and
provider names.

- Characters in range 0x01-0x1f except '\t', '\n', and '\r' are replaced
with '?'. Those characters are disallowed in XML.
- '&', '<', '>', '\'', '"' and characters in range 0x7f-0xff are
replaced with XML numeric character reference.

If the kern.geom.confxml sysctl provides invalid XML, libgeom
geom_xml2tree() fails and utilities using it do not work. Unsafe
characters are common in msdosfs and cd9660 labels.

PR: kern/104389

Escape characters unsafe for XML output in GEOM class, instance and
provider names.

- Characters in range 0x01-0x1f except '\t', '\n', and '\r' are replaced
with '?'. Those characters are disallowed in XML.
- '&', '<', '>', '\'', '"' and characters in range 0x7f-0xff are
replaced with XML numeric character reference.

If the kern.geom.confxml sysctl provides invalid XML, libgeom
geom_xml2tree() fails and utilities using it do not work. Unsafe
characters are common in msdosfs and cd9660 labels.

PR: kern/104389
Submitted by: Doug Steinwand (original version)
Reviewed by: pjd
Discussed on: freebsd-geom
MFC after: 3 weeks

Expose stripe offset and stripe size through libgeom and geom(8) userland
utilities.

Reviewed by: pjd, mav (earlier version)

- Import the HEAD csup code which is the basis for the cvsmode work.

Make the kern.geom.conftxt sysctl more usable by also dumping the
MD class. Previously only the DISK class was dumped. The only
consumer of this sysctl is libdisk (i.e. sysinstall) and it tests
explicitly for instances of the DISK class. Dumping other classes
is therefore harmless.
By also dumping the MD class regression tests can be written that
use the MD class for operations that would normally be done on the
DISK class. The sysctl can now be used to test if those operations
took an effect. An example is partitioning.

Rearrange some of the GEOM debugging tools to be more structured.

Retire g_sanity() and corresponding debugflag (0x8)

Retire g_{stall,release}_events().

Under #ifdef DIAGNOSTIC:

Make g_valid_obj() an official function and have it return an an
non-zero integer which indicates the kind of object when found.

Implement G_VALID_{CLASS,GEOM,CONSUMER,PROVIDER}() macros based
on g_valid_obj().

Sprinkle calls to these macros liberally over the infrastructure.

Always check that we do not free a live object.

Re-arrange and consolidate some random debugging stuff

Handle geoms which are withering away specially in the dump functions.

Use __FBSDID().

Approved by: phk

Add missing va_end() calls.

Noticed by: tmm

Introduce a g_waitfor_event() function which posts an event and waits for
it to be run (or cancelled) and use this instead of home-rolled versions.

Time has run from the "run GEOM in userland" harness, and the new regression
test is built to test GEOM as running in the kernel.

This commit is basically "unifdef -D_KERNEL" to remove the mainly #include
related code to support the userland-harness.

Add handling for cancelled events in the g_call_me() methods.

Change events to have an array of "void *" references, and give the
event posting functions varargs to fill these.

Attribute g_call_me() to appropriate g_geom's where necessary.

Add a flag argument to g_call_me() methods which will be used to signal
cancellation of events in the future.

This commit should be a no-op.

Including <sys/stdint.h> is (almost?) universally only to be able to use
%j in printfs, so put a newsted include in <sys/systm.h> where the printf
prototype lives and save everybody else the trouble.

Commit the correct copy of the g_stat structure.

Add debug.sizeof.g_stat sysctl.

Set the id field of the g_stat when we create consumers and providers.

Remove biocount from consumer, we will use the counters in the g_stat
structure instead. Replace one field which will need to be atomically
manipulated with two fields which will not (stat.nop and stat.nend).

Change add companion field to bio_children: bio_inbed for the exact
same reason.

Don't output the biocount in the confdot output.

Fix KASSERT in g_io_request().

Add sysctl kern.geom.collectstats defaulting to off.

Collect the following raw statistics conditioned on this sysctl:

for each consumer and provider {
total number of operations started.
total number of operations completed.
time last operation completed.
sum of idle-time.
for each of BIO_READ, BIO_WRITE and BIO_DELETE {
number of operations completed.
number of bytes completed.
number of ENOMEM errors.
number of other errors.
sum of transaction time.
}
}

API for getting hold of these statistics data not included yet.

Constification and some s/int/u_int/ changes.

Do not call the dumpconf method unless there is one.
Compare pointers with NULL.

Partially submitted by: Christian Carstensen <cc@gate5.de>
Approved by: re

Remove harmless but irritating printf.

Make this compile in the userland shims again.

Sponsored by: DARPA & NAI Labs

Add the remaning part of the new libdisk interaction.

WARNING: This is not a published interface, it is a stopgap measure for
WARNING: libdisk so we can get 5.0-R out of the door.

Sponsored by: DARPA & NAI Labs

Make the sectorsize a property of providers so we can include it in the XML
output.

Sponsored by: DARPA & NAI Labs

Use %jd instead of %lld now that we have it.

Add more KASSERTS.

Sponsored by: DARPA & NAI Labs.

Properly isolate the locking domains of sysctl from the topology lock
for the sysctls which report the configuration.

Sponsored by: DARPA & NAI Labs.

Retire g_io_fail() and let g_io_deliver() take an error argument instead.

Sponsored by: DARPA & NAI Labs.

Remove "magicspace". It looks good on paper, it doesn't work in practice.

Sponsored by: DARPA & NAI Labs.

Introduce the concept of "magic spaces", and implement them in most of
the relevant classes.

Some methods may implement various "magic spaces", this is reserved
or magic areas on the disk, set a side for various and sundry purposes.
A good example is the BSD disklabel and boot code on i386 which occupies
a total of four magic spaces: boot1, the disklabel, the padding behind
the disklabel and boot2. The reason we don't simply tell people to
write the appropriate stuff on the underlying device is that (some of)
the magic spaces might be real-time modifiable. It is for instance
possible to change a disklabel while partitions are open, provided
the open partitions do not get trampled in the process.

Sponsored by: DARPA & NAI Labs.

Improve the cross-references in the XML output.

Explained by: des
Sponsored by: DARPA & NAI Labs.

Introduce some serious paranoia to try to catch a memory overwrite problem
as early as possible.

Sponsored by: DARPA & NAI Labs

Make life easier for reference-vector generatorts in tools/regression/geom
by including a FreeBSD friendly CVS identifier in the XML output.

Sponsored by: DARPA & NAI Labs.

s/classs/classes/ to fixup grammer after the previous global renaming.

Sponsored by: DARPA & NAI Labs

In the absense of any smarter way to do this, cast various printf
arguments to silence printf format warnings.

Eliminate some thread pointers which do not make sense anymore.

Split private parts of geom.h into geom_int.h. The latter should
never be included in class implemtations.

Cave in to tradition and rename "methods" to "classes".

First commit of the GEOM subsystem to make it easier for people to
test and play with this.

This is not yet production quality and should be run only on dedicated
test boxes.

For people who want to develop transformations for GEOM there exist a
set of shims to run geom in userland (ask phk@freebsd.org).

Reports of all kinds to: phk@freebsd.org
Please include in report:
dmesg
sysctl debug.geomdot
sysctl debug.geomconf

Known significant limitations:
no kernel dump facility.
ioctls severely restricted.

Sponsored by: DARPA, NAI Labs