bectl: Simplify command aliases.

MFC after: 3 days
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D44406

bectl: Use geopt() and drop mention of -?.

MFC after: 3 days
PR: 272260
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D44405

sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bectl: make mount subcommand less verbose

The mount subcommand currently produces output such as:
# bectl mount <bootenv>
Successfully mounted <bootenv> at <mountpoint>

This commit changes it to only print the mountpoint:
# bectl mount <bootenv>
<mountpoint>

This makes it easier to script the mount subcommand. If an error occurs
while mounting, an error message is printed to stderr and bectl will
exit with a non-zero value.

PR: 273180
Reviewed by: kevans, asomers
Differential Revision: https://reviews.freebsd.org/D41562

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

bectl: document the -r flag to bectl

Independent of all of the commands, bectl itself takes an `-r` flag that
specifies the BE root to use. This was originally added to facilitate
testing, but it was later discovered to be incredibly useful in other
scenarios; e.g., trying to recover some boot environments in rescue
media.

The "BE root" described here is the parent dataset that holds boot
environments, but I've no idea if that's an accepted definition for that
dataset.

Reviewed by: gallatin, imp, Pau Amma
MFC after: 1 week
Differential Review: https://reviews.freebsd.org/D39710

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from: OpenZFS
OpenZFS commit: 431083f75bdd3efaee992bdd672625ec7240d252

bectl: Improve error message when ZFS root is not found.

When recovering a system that is unbootable due to some
problem with the active BE, it is likely you'll be booted
from a rescue image running UFS. In this case, bectl
needs help finding the zpool root that you want to operate
on. In this case, improve the error message to suggest
specifying a root, rather than just emitting a generic
error message that might imply, to the naive user, that
there is a ZFS compatibility issue between the rescue
image and the on-disk ZFS pool.

Reviewed by: imp, kevans
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D39346

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.

While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.

Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605

bectl(8): don't allow creation of boot environments with spaces

Boot environment datasets that contain spaces are not bootable.

When a user attempts to create a boot environment with a space, abort
the creation and print an error message.

PR: 254441
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D30194

bectl: remove spurious aok variable

This rode in with the OpenZFS import. It may have been necessary at some
point, but it is no longer and it breaks the WITHOUT_DYNAMICROOT build as
it collides with the definition in libspl.

Reported-by: Michael Dexter

bectl(8): sync man page and help text

Sync man page with behavior of bectl(8).

Sync help text with man page.

PR: 246697
Reported by: olgeni
Submitted by: olgeni (with changes)
Reviewed by: kevans, olgeni
Approved by: kevans (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27482

Fix r368197: suppress error printing for the "check" command.

Reviewed by: kevans

Print at least something when failing.

bectl(8): missing flag in the help message

Flag -o for destroy subcommand is missing in the help message,
but present in the man page. Fix it.

PR: 249325
Approved by: kevans
Differential Revision: https://reviews.freebsd.org/D26429

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.

Capitalize some user-visible output messages in
the bectl utility.

No functional changes.

Approved by: imp@
MFC after: 7 days
Differential Revision: https://reviews.freebsd.org/D22330

bectl(8): destroy: use BE_DESTROY_AUTOORIGIN if -o is not specified

-o will force the origin to be destroyed unconditionally.
BE_DESTROY_AUTOORIGIN, on the other hand, will only destroy the origin if it
matches the format used by be_snapshot. This lets us clean up the snapshots
that are clearly not user-managed (because we're creating them) while
leaving user-created snapshots in place and warning that they're still
around when the BE created goes away.

bectl(8): Add a "check" command

This command simply returns 0 at the moment and explicitly takes no
arguments. This should be used by utilities wanting to see if bectl can
operate on the system they're running, or with a specific root (`bectl -r`).
It may grow more checks than "will libbe successfully init" in the future,
but for now this is enough as that checks for the dataset mounted at "/" and
that it looks capable of being a BE root (e.g. it's not a top-level dataset)

bectl commands can now specify if they want to be silent, and this will turn
off libbe_print_on_error so they can control the output as needed. This is
already used in `bectl check`, and may be turned on in the future for some
other commands where libbe errors are better suppressed as the failure mode
may be obvious.

Requested by: David Fullard
MFC after: 3 days

bectl(8): implement sorting for 'bectl list' output

Allow 'bectl list' to sort output by a given property name. The property
name is passed in using a command-line flag, '-c' for ascending order and
'-C' for descending order. The properties allowed to sort by are:

- name (the default output, even if '-c' or '-C' are not used)
- creation
- origin
- used
- usedds
- usedsnap
- usedrefreserv

The default output for 'bectl list' is now ascending alphabetical order of
BE name.

To sort by creation time from earliest to latest, the command would be
'bectl list -c creation'

Submitted by: Rob Fairbanks <rob.fx907 gmail com>
Reviewed by: ler
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20818

bectl(8): create non-recursive boot environments

bectl advertises that it has the ability to create recursive and
non-recursive boot environments. This patch implements that functionality
using the be_create_depth API provided by libbe. With this patch, bectl now
works as bectl(8) describes in regards to creating recursive/non-recursive
boot environments.

Submitted by: Rob Fairbanks <rob.fx907 gmail com> (with minor changes)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20240

bectl(8): Add -o flag to destroy to clean up the origin snapshot of BE

We can't predict when destruction of origin is needed, and currently we have
a precedent for not prompting for things. Leave the decision up to the user
of bectl(8) if they want the origin snapshot to be destroyed or not.

Emits a warning when -o isn't used and an origin snapshot is left to be
cleaned up, for the time being. This is handy when one drops the -o flag but
really did want to clean up the origin.

A couple of -e ignore's have been sprinkled around the test suite for places
that we don't care that the origin's not been cleaned up. -o functionality
tests will be added in the future, but are omitted for now to reduce
conflicts with work in flight to fix bits of the tests.

Reported by: Shawn Webb
MFC after: 1 week

libbe(3): Change be_mount to mount/unmount child datasets

This set of changes is geared towards making bectl respect deep boot
environments when they exist and are mounted. The deep BE composition
functionality (`bectl add`) remains disabled for the time being. This set of
changes has no effect for the average user. but allows deep BE users to
upgrade properly with their current setup.

libbe(3): Open the target boot environment and get a zfs handle, then pass
that with the target mountpoint to be_mount_iter; If the BE_MNT_DEEP flag is
set call zfs_iter_filesystems and mount the child datasets.

Similar logic is employed when unmounting the datasets, save for children
are unmounted first.

bectl(8): Change bectl_cmd_jail to pass the BE_MNT_DEEP flag when
calling be_mount as well as call be_unmount when cleaning up after the
jail has exited instead of umount(2) directly.

PR: 234795
Submitted by: Wes Maag <jwmaag_gmail.com> (test additions by kevans)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18796

bectl: sync usage with man page, removing stray multibyte characters
in the process.

PR: 233526
Submitted by: tigersharke@gmail.com (original version)
Reviewed by: kevans
Approved by: kib (mentor, implicit)
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D18335

bectl(3)/libbe(3): Allow BE root to be specified

Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:

- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)

Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.

bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt

this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.

MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029

libbe(3)/bectl(8): Make consistent with beadm

vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.

Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.

A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.

Approved by: re (kib)

bectl(8): Man page and usage cleanup

- Some overly-long lines
- Consistently using .Brq ({})
- Consistently using .Cm
- Not using .Ao/.Ac around .Ar

PR: 230576
Submitted by: Yuri Pankov (with a fair amount of rebasing pre-commit)

bectl(8): jail: Tear down jail by default after command exits

Add a -U flag to get back the old behavior. The new behavior is a little
more friendly to the common use cases, jail the BE and execute a script.
Having the jail torn down automatically when the script is finished, or when
you exit the shell, is a little more friendly than having to remember to
`bectl ujail`.

Batch mode (-b) will continue to leave the jail up, as it's assumed the
caller has other intentions.

Submitted by: Shawn Webb (partially)

bectl(8): Allow running a custom command in the 'jail' subcommand

Instead of always running /bin/sh, allow the user to specify the command
to run. The jail is not removed when the command finishes. Meaning,
`bectl unjail` will still need to be run.

For example:

```
bectl jail newBE pkg upgrade
bectl ujail newBE
```

Submitted by: Shawn Webb
Obtained from: HardenedBSD (8b451014ab)

bectl(8): Add batch mode to jail subcommand

Adding batch mode to the jail `bectl(8)` subcommand enables jailing of
ZFS Boot Environments in a scriptable fashion.

Submitted by: Shawn Webb
Obtained from: HardenedBSD (9e72d1c59a and ef7b6d9e1c with minor edit)

bectl(8): Kit-kat bar

bectl(8): Implement the 'create a snapshot' variant of create

bectl(8): Rename "index" variable, which shadows a global in some lands

libbe(3)/bectl(8): Kill off the 'add' functionality for now

The mostly-undocumented 'add' functionality, from initial read-through, is
intended for construction of deep ("bdrewery style") boot environments.
However, it's mostly broken at this point. `#if SOON` it out on both sides
so that we're not exposing a broken API/feature.

Work will resume on it in due time.

libbe(3)/bectl(8): Standardize $FreeBSD$ IDs

libbe(3)/bectl(8): Standardize copyright headers

- File names don't necessarily need to be repeated
- Add SPDX tags
- Add a missing copyright for Kyle Kneitinger in bectl.8, originally written
by him in GSoC 2017; his standard copyright notice has been copied from
other files within the same directory to remain consistent with how he
clearly wished to portray it

bectl(8): Provide -u option to unset jail parameters

All but name, host.hostname, and path may be completely unset.

bectl(8): Split list functionality out into its own file as well

bectl(8): bectl jail improvements

- Support passing arbitrary jail arguments via -o
- Split the related (and rewritten since the GSoC) jail bits out into a new
bectl_jail.c file, to reduce clutter in bectl.c
- Don't use RFC 1918 IP space [0]; we'll instead set no default IPv4 and let
the user pass in any address options they wish via -o

Reported by: rgrimes [0], Shawn Webb [0]

bectl(8): Some light cleanup and commenting

bectl(8): Implement `bectl list -s`

be_get_dataset_snapshots has been added to libbe(3), effectively returning
the same information as be_get_bootenv_props but for snapshots of the given
dataset. The assumption is that one will have the BE dataset name before
wanting to grab snapshots.

bectl: Implement -D ("space if origin datasets were deleted")

This also accomplishes the following:

- Proxy through zfs_nicenum as be_nicenum, because it looks better than
humanize_number and would presumably be useful to other libbe consumers.

- Rename be_get_snapshot_props to be_get_dataset_props, make it more useful

bectl(8): Take -H parameter to list into account

-H is for a scripted format, where all fields are tab-delimited and the
headers go away. We accomplish this by splitting out pad printing to a
separate function that'll take into account whether we're supposed to be
scripted or not.

This has the nice side effect of maintaining positive column sizes again.

bectl(8): Sort BEs lightly by active (now or later) BE, then others

While it could be preferred to do this at insert in libbe(3), there's no
convenient way to insert at the head of an nvlist. Instead, we'll make two
passes over- once to print anything active either now or at nextboot, and
another to print everything else.

This doesn't actually impact performance in a significant way here, so we'll
worry about further optimizations if the need actually arises.

bectl(8): Add some relevant `bectl list -a` information

bectl list -a should show the boot environment, its relevant dataset, and
the snapshot it was created from. The header also changes to indicate the
rough order in which these things will show.

While here, start doing a little bit of spring cleaning and splitting
different bits out.

bectl(8): Take origin snapshot into account when calculating used space

This more closely matches the behavior for beadm. The associated libbe(3)
API is still getting worked out a little bit.

bectl(8): Just use a single pointer for all string props

bectl(8): Move dataset printing out into its own function

This may later get reused for printing snapshot and/or origin snapshot
information.

bectl(8): Remove sneaky comment

bectl(8): Hack together a more proper `bectl list`

Note that the space is currently just the 'used' property of the dataset.

bectl(8): Remove the jail mount point with unjail

bectl(8): Redo jail using jail(3) API

The jail is created with allow.mount, allow.mount.devfs, and
enforce_statfs=1. Upon creation, we immediately attach, chdir to "/", and
drop the user into a shell inside the jail.

The default IP for this is arbitrarily 10.20.30.40.

bectl(8): Support unjailing a boot environment

The given parameter may either be a jid, jail name, or a BE name. In all
cases, the parameter will be resolved to a jid and bectl(8) will
sanity-check that there's actually a BE mounted at the requested jail root
before invoking jail_remove(2).

libbe(3)/bectl(8): Provide and use proper alloc/free for property lists

libbe(3)/be(8): Drop WARNS overrides, fix all fallout

Based on the idea that we shouldn't have all-new library and utility going
into base that need WARNS=1...

- Decent amount of constification
- Lots of parentheses
- Minor other nits

bectl(8): Start dumping out BE information with `bectl list`

For the moment, this is a primitive nvlist dump of what we get back from
be_get_bootenv_props as a proof-of-concept and to make sure that we're
getting back the kind of information we want to see from list.

bectl(8): Consistent style

bectl(8): Rename error messages

bectl.c: Chase rename

Rename be(1) to bectl(8); continues to live in /sbin

Discussed with: rpokala, allanjude

bectl: Improve error message when ZFS root is not found.

When recovering a system that is unbootable due to some
problem with the active BE, it is likely you'll be booted
from a rescue image running UFS. In this case, bectl
needs help finding the zpool root that you want to operate
on. In this case, improve the error message to suggest
specifying a root, rather than just emitting a generic
error message that might imply, to the naive user, that
there is a ZFS compatibility issue between the rescue
image and the on-disk ZFS pool.

Reviewed by: imp, kevans
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D39346

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.

While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.

Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605

bectl(8): don't allow creation of boot environments with spaces

Boot environment datasets that contain spaces are not bootable.

When a user attempts to create a boot environment with a space, abort
the creation and print an error message.

PR: 254441
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D30194

bectl: remove spurious aok variable

This rode in with the OpenZFS import. It may have been necessary at some
point, but it is no longer and it breaks the WITHOUT_DYNAMICROOT build as
it collides with the definition in libspl.

Reported-by: Michael Dexter

bectl(8): sync man page and help text

Sync man page with behavior of bectl(8).

Sync help text with man page.

PR: 246697
Reported by: olgeni
Submitted by: olgeni (with changes)
Reviewed by: kevans, olgeni
Approved by: kevans (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27482

Fix r368197: suppress error printing for the "check" command.

Reviewed by: kevans

Print at least something when failing.

bectl(8): missing flag in the help message

Flag -o for destroy subcommand is missing in the help message,
but present in the man page. Fix it.

PR: 249325
Approved by: kevans
Differential Revision: https://reviews.freebsd.org/D26429

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.

Capitalize some user-visible output messages in
the bectl utility.

No functional changes.

Approved by: imp@
MFC after: 7 days
Differential Revision: https://reviews.freebsd.org/D22330

bectl(8): destroy: use BE_DESTROY_AUTOORIGIN if -o is not specified

-o will force the origin to be destroyed unconditionally.
BE_DESTROY_AUTOORIGIN, on the other hand, will only destroy the origin if it
matches the format used by be_snapshot. This lets us clean up the snapshots
that are clearly not user-managed (because we're creating them) while
leaving user-created snapshots in place and warning that they're still
around when the BE created goes away.

bectl(8): Add a "check" command

This command simply returns 0 at the moment and explicitly takes no
arguments. This should be used by utilities wanting to see if bectl can
operate on the system they're running, or with a specific root (`bectl -r`).
It may grow more checks than "will libbe successfully init" in the future,
but for now this is enough as that checks for the dataset mounted at "/" and
that it looks capable of being a BE root (e.g. it's not a top-level dataset)

bectl commands can now specify if they want to be silent, and this will turn
off libbe_print_on_error so they can control the output as needed. This is
already used in `bectl check`, and may be turned on in the future for some
other commands where libbe errors are better suppressed as the failure mode
may be obvious.

Requested by: David Fullard
MFC after: 3 days

bectl(8): implement sorting for 'bectl list' output

Allow 'bectl list' to sort output by a given property name. The property
name is passed in using a command-line flag, '-c' for ascending order and
'-C' for descending order. The properties allowed to sort by are:

- name (the default output, even if '-c' or '-C' are not used)
- creation
- origin
- used
- usedds
- usedsnap
- usedrefreserv

The default output for 'bectl list' is now ascending alphabetical order of
BE name.

To sort by creation time from earliest to latest, the command would be
'bectl list -c creation'

Submitted by: Rob Fairbanks <rob.fx907 gmail com>
Reviewed by: ler
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20818

bectl(8): create non-recursive boot environments

bectl advertises that it has the ability to create recursive and
non-recursive boot environments. This patch implements that functionality
using the be_create_depth API provided by libbe. With this patch, bectl now
works as bectl(8) describes in regards to creating recursive/non-recursive
boot environments.

Submitted by: Rob Fairbanks <rob.fx907 gmail com> (with minor changes)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20240

bectl(8): Add -o flag to destroy to clean up the origin snapshot of BE

We can't predict when destruction of origin is needed, and currently we have
a precedent for not prompting for things. Leave the decision up to the user
of bectl(8) if they want the origin snapshot to be destroyed or not.

Emits a warning when -o isn't used and an origin snapshot is left to be
cleaned up, for the time being. This is handy when one drops the -o flag but
really did want to clean up the origin.

A couple of -e ignore's have been sprinkled around the test suite for places
that we don't care that the origin's not been cleaned up. -o functionality
tests will be added in the future, but are omitted for now to reduce
conflicts with work in flight to fix bits of the tests.

Reported by: Shawn Webb
MFC after: 1 week

libbe(3): Change be_mount to mount/unmount child datasets

This set of changes is geared towards making bectl respect deep boot
environments when they exist and are mounted. The deep BE composition
functionality (`bectl add`) remains disabled for the time being. This set of
changes has no effect for the average user. but allows deep BE users to
upgrade properly with their current setup.

libbe(3): Open the target boot environment and get a zfs handle, then pass
that with the target mountpoint to be_mount_iter; If the BE_MNT_DEEP flag is
set call zfs_iter_filesystems and mount the child datasets.

Similar logic is employed when unmounting the datasets, save for children
are unmounted first.

bectl(8): Change bectl_cmd_jail to pass the BE_MNT_DEEP flag when
calling be_mount as well as call be_unmount when cleaning up after the
jail has exited instead of umount(2) directly.

PR: 234795
Submitted by: Wes Maag <jwmaag_gmail.com> (test additions by kevans)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18796

bectl: sync usage with man page, removing stray multibyte characters
in the process.

PR: 233526
Submitted by: tigersharke@gmail.com (original version)
Reviewed by: kevans
Approved by: kib (mentor, implicit)
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D18335

bectl(3)/libbe(3): Allow BE root to be specified

Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:

- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)

Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.

bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt

this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.

MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029

libbe(3)/bectl(8): Make consistent with beadm

vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.

Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.

A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.

Approved by: re (kib)

bectl(8): Man page and usage cleanup

- Some overly-long lines
- Consistently using .Brq ({})
- Consistently using .Cm
- Not using .Ao/.Ac around .Ar

PR: 230576
Submitted by: Yuri Pankov (with a fair amount of rebasing pre-commit)

bectl(8): jail: Tear down jail by default after command exits

Add a -U flag to get back the old behavior. The new behavior is a little
more friendly to the common use cases, jail the BE and execute a script.
Having the jail torn down automatically when the script is finished, or when
you exit the shell, is a little more friendly than having to remember to
`bectl ujail`.

Batch mode (-b) will continue to leave the jail up, as it's assumed the
caller has other intentions.

Submitted by: Shawn Webb (partially)

bectl(8): Allow running a custom command in the 'jail' subcommand

Instead of always running /bin/sh, allow the user to specify the command
to run. The jail is not removed when the command finishes. Meaning,
`bectl unjail` will still need to be run.

For example:

```
bectl jail newBE pkg upgrade
bectl ujail newBE
```

Submitted by: Shawn Webb
Obtained from: HardenedBSD (8b451014ab)

bectl(8): Add batch mode to jail subcommand

Adding batch mode to the jail `bectl(8)` subcommand enables jailing of
ZFS Boot Environments in a scriptable fashion.

Submitted by: Shawn Webb
Obtained from: HardenedBSD (9e72d1c59a and ef7b6d9e1c with minor edit)

bectl(8): Kit-kat bar

bectl(8): Implement the 'create a snapshot' variant of create

bectl(8): Rename "index" variable, which shadows a global in some lands

libbe(3)/bectl(8): Kill off the 'add' functionality for now

The mostly-undocumented 'add' functionality, from initial read-through, is
intended for construction of deep ("bdrewery style") boot environments.
However, it's mostly broken at this point. `#if SOON` it out on both sides
so that we're not exposing a broken API/feature.

Work will resume on it in due time.

libbe(3)/bectl(8): Standardize $FreeBSD$ IDs

libbe(3)/bectl(8): Standardize copyright headers

- File names don't necessarily need to be repeated
- Add SPDX tags
- Add a missing copyright for Kyle Kneitinger in bectl.8, originally written
by him in GSoC 2017; his standard copyright notice has been copied from
other files within the same directory to remain consistent with how he
clearly wished to portray it

bectl(8): Provide -u option to unset jail parameters

All but name, host.hostname, and path may be completely unset.

bectl(8): Split list functionality out into its own file as well

bectl(8): bectl jail improvements

- Support passing arbitrary jail arguments via -o
- Split the related (and rewritten since the GSoC) jail bits out into a new
bectl_jail.c file, to reduce clutter in bectl.c
- Don't use RFC 1918 IP space [0]; we'll instead set no default IPv4 and let
the user pass in any address options they wish via -o

Reported by: rgrimes [0], Shawn Webb [0]

bectl(8): Some light cleanup and commenting

bectl(8): Implement `bectl list -s`

be_get_dataset_snapshots has been added to libbe(3), effectively returning
the same information as be_get_bootenv_props but for snapshots of the given
dataset. The assumption is that one will have the BE dataset name before
wanting to grab snapshots.

bectl: Implement -D ("space if origin datasets were deleted")

This also accomplishes the following:

- Proxy through zfs_nicenum as be_nicenum, because it looks better than
humanize_number and would presumably be useful to other libbe consumers.

- Rename be_get_snapshot_props to be_get_dataset_props, make it more useful

bectl(8): Take -H parameter to list into account

-H is for a scripted format, where all fields are tab-delimited and the
headers go away. We accomplish this by splitting out pad printing to a
separate function that'll take into account whether we're supposed to be
scripted or not.

This has the nice side effect of maintaining positive column sizes again.

bectl(8): Sort BEs lightly by active (now or later) BE, then others

While it could be preferred to do this at insert in libbe(3), there's no
convenient way to insert at the head of an nvlist. Instead, we'll make two
passes over- once to print anything active either now or at nextboot, and
another to print everything else.

This doesn't actually impact performance in a significant way here, so we'll
worry about further optimizations if the need actually arises.

bectl(8): Add some relevant `bectl list -a` information

bectl list -a should show the boot environment, its relevant dataset, and
the snapshot it was created from. The header also changes to indicate the
rough order in which these things will show.

While here, start doing a little bit of spring cleaning and splitting
different bits out.

bectl(8): Take origin snapshot into account when calculating used space

This more closely matches the behavior for beadm. The associated libbe(3)
API is still getting worked out a little bit.

bectl(8): Just use a single pointer for all string props

bectl(8): Move dataset printing out into its own function

This may later get reused for printing snapshot and/or origin snapshot
information.

bectl(8): Remove sneaky comment

bectl(8): Hack together a more proper `bectl list`

Note that the space is currently just the 'used' property of the dataset.

bectl(8): Remove the jail mount point with unjail

bectl(8): Redo jail using jail(3) API

The jail is created with allow.mount, allow.mount.devfs, and
enforce_statfs=1. Upon creation, we immediately attach, chdir to "/", and
drop the user into a shell inside the jail.

The default IP for this is arbitrarily 10.20.30.40.

bectl(8): Support unjailing a boot environment

The given parameter may either be a jid, jail name, or a BE name. In all
cases, the parameter will be resolved to a jid and bectl(8) will
sanity-check that there's actually a BE mounted at the requested jail root
before invoking jail_remove(2).

libbe(3)/bectl(8): Provide and use proper alloc/free for property lists

libbe(3)/be(8): Drop WARNS overrides, fix all fallout

Based on the idea that we shouldn't have all-new library and utility going
into base that need WARNS=1...

- Decent amount of constification
- Lots of parentheses
- Minor other nits

bectl(8): Start dumping out BE information with `bectl list`

For the moment, this is a primitive nvlist dump of what we get back from
be_get_bootenv_props as a proof-of-concept and to make sure that we're
getting back the kind of information we want to see from list.

bectl(8): Consistent style

bectl(8): Rename error messages

bectl.c: Chase rename

Rename be(1) to bectl(8); continues to live in /sbin

Discussed with: rpokala, allanjude