History log of /freebsd-current/lib/libbe/be.c
Revision Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
# b139f2ee 02-Feb-2024 rilysh <nightquick@proton.me>

lib/libbe/be.c: remove an extra semicolon

Signed-off-by: rilysh <nightquick@proton.me>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/959


# 5086b6ec 26-Jan-2024 R. Christian McDonald <rcm@FreeBSD.org>

libbe: handle destroying/renaming temporary/bootonce boot environments

When a temporary/bootonce boot environment is renamed, we need to also
update the bootenv nvlist on-disk to reflect the new name. Additionally,
when a temporary/bootonce boot environment is destroyed, we also need to
clear out the on-disk state.

Reviewed by: kevans
Approved by: kp
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D43591


# a2f733ab 24-Nov-2023 Warner Losh <imp@FreeBSD.org>

lib: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix


# 1d386b48 16-Aug-2023 Warner Losh <imp@FreeBSD.org>

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/


# 4b426cf3 17-Jul-2023 R. Christian McDonald <rcm@rcm.sh>

libbe: recursively promote deep BE datasets

beadm will recursively promote deep BE datasets. In order to match the
beadm behavior, we need to recursively iterate over child filesystems
and promote them along the way.

This patch further refines the work from D40903, completing the fix for
promotion.

Reviewed by: kevans, rew
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D40972


# 5d082601 10-Jul-2023 Kyle Evans <kevans@FreeBSD.org>

libbe: promote activated BEs all the way

This matches the beadm behavior; generally, we need to keep promoting
until the BE is no longer a clone from a snapshot. This fixes scenarios
where the dataset associated with a BE's origin is itself a clone,
activating the BE previously would promote it to a clone of the origin's
origin.

We could keep using be_get_dataset_props here, except for two
annoyances:

1.) I couldn't find a clean way to just clear an nvlist rather than
having to re-alloc it, and I didn't want to just remove the one prop
we're inspecting out of it.

2.) That's a lot of overhead when all we want to do is fetch the origin
anyways.

Note that this is not a complete fix, but it does fix the majority of
cases; deep BE subordinates are still notably broken, pending a patch
from Christian.

Reported by: R. Christian McDonald <rcm@rcm.sh>
Reviewed by: rew
Differential Revision: https://reviews.freebsd.org/D40903


# 2f700ca9 06-Jul-2023 Kyle Evans <kevans@FreeBSD.org>

libbe: bail out early if the zfs kmod isn't loaded

As noted in the comment, we already know the rest of libbe_init() will
fail because there's no pool imported. Avoid the side effect by
checking beforehand and bailing out early.

With this, freebsd-update(8) should no longer trigger a load of the zfs
kmod just because it runs `bectl check`.

Reviewed by: jwmaag_gmail.com, rew
Differential Revision: https://reviews.freebsd.org/D36188


# 4d846d26 10-May-2023 Warner Losh <imp@FreeBSD.org>

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix


# 2a58b312 03-Apr-2023 Martin Matuska <mm@FreeBSD.org>

zfs: merge openzfs/zfs@431083f75

Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"

Obtained from: OpenZFS
OpenZFS commit: 431083f75bdd3efaee992bdd672625ec7240d252


# 8e933d9c 14-Mar-2023 John Grafton <john.grafton@gmail.com>

libbe: Avoid double printing cloning errors.

be_clone calls be_clone_cb and both call set_error on the return
error path. set_error prints the error resulting in a double print.
be_clone_cb should just return the error code and allow be_clone
to print it.

PR: 265248
Reported by: Graham Perrin
Reviewed by: imp, kevans
Pull Request: https://github.com/freebsd/freebsd-src/pull/685

# 15f0b8c3 25-Jan-2023 Martin Matuska <mm@FreeBSD.org>

zfs: merge openzfs/zfs@9cd71c860 (master)

Notable upstream pull request merges:
#13805 Configure zed's diagnosis engine with vdev properties
#14110 zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
#14121 Batch enqueue/dequeue for bqueue
#14123 arc_read()/arc_access() refactoring and cleanup
#14159 Bypass metaslab throttle for removal allocations
#14243 Implement uncached prefetch
#14251 Cache dbuf_hash() calculation
#14253 Allow reciever to override encryption property in case of replication
#14254 Restrict visibility of per-dataset kstats inside FreeBSD jails
#14255 Zero end of embedded block buffer in dump_write_embedded()
#14263 Cleanups identified by CodeQL and Coverity
#14264 Miscellaneous fixes
#14272 Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names
#14287 FreeBSD: Remove stray debug printf
#14288 Colorize zfs diff output
#14289 deadlock between spa_errlog_lock and dp_config_rwlock
#14291 FreeBSD: Fix potential boot panic with bad label
#14292 Add tunable to allow changing micro ZAP's max size
#14293 Turn default_bs and default_ibs into ZFS_MODULE_PARAMs
#14295 zed: add hotplug support for spare vdevs
#14304 Activate filesystem features only in syncing context
#14311 zpool: do guid-based comparison in is_vdev_cb()
#14317 Pack zrlock_t by 8 bytes
#14320 Update arc_summary and arcstat outputs
#14328 FreeBSD: catch up to 1400077
#14376 Use setproctitle to report progress of zfs send
#14340 Remove some dead ARC code
#14358 Wait for txg sync if the last DRR_FREEOBJECTS might result in a hole
#14360 libzpool: fix ddi_strtoull to update nptr
#14364 Fix unprotected zfs_znode_dmu_fini
#14379 zfs_receive_one: Check for the more likely error first
#14380 Cleanup of dead code suggested by Clang Static Analyzer
#14397 Avoid passing an uninitialized index to dsl_prop_known_index
#14404 Fix reading uninitialized variable in receive_read
#14407 free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report
#14418 Introduce minimal ZIL block commit delay
#14422 x86 assembly: fix .size placement and replace .align with .balign

Obtained from: OpenZFS
OpenZFS commit: 9cd71c8604d52def22ffaddc35755712f0fb9349


# dadb9c70 02-Apr-2022 Kyle Evans <kevans@FreeBSD.org>

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.

While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.

Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605

# 7edc1bd9 08-Jan-2021 Gleb Smirnoff <glebius@FreeBSD.org>

When be_activate() turns on a new boot environment, it always deactivates
the current one first. And if it fails to do so, it abandons activation.
However, with the new bootonce feature, there is a legitimate case when
a pool doesn't have "bootfs" property set. Check for this case before
calling be_deactivate().

Reviewed by: kevans

# e307eb94 21-Sep-2020 Toomas Soome <tsoome@FreeBSD.org>

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

# eac7052f 04-Sep-2020 Matt Macy <mmacy@FreeBSD.org>

ZFS: MFV 2.0-rc1-gfd20a8

- fixes jail delegation
- fixes raw kstat unsupported complaints
- exposes dbgmsg, dbuf, and checksum benchmark stats
- restore rename -u support


# 9e5787d2 24-Aug-2020 Matt Macy <mmacy@FreeBSD.org>

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

# 9d6d8bf8 21-Jul-2020 Kyle Evans <kevans@FreeBSD.org>

libbe: annotate lbh as __unused in be_is_auto_snapshot_name

lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a post-OpenZFS-import world will be able to raise WARNS of
libbe to the default (pending some minor changes to openzfs libzfs).

MFC after: 3 days

# 6966ac05 23-Jan-2020 Kyle Evans <kevans@FreeBSD.org>

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.

# 8f5c6c31 02-Jan-2020 Kyle Evans <kevans@FreeBSD.org>

libbe(3): promote dependent clones when destroying an environment

When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.

This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.

PR: 242592
Submitted by: Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22953

# 485172f5 06-Dec-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: fix build against sysutils/openzfs, part 1

This is the half of the changes required that work as-is with both in-tree
ZFS and the new hotness, sysutils/openzfs. Highlights are less dependency
on header pollution (from somewhere) and using 'mnttab' instead of
'extmnttab'. In the in-tree ZFS, the latter is a #define for the former,
but in the port extmnttab is actually a distinct struct that's a super-set
of mnttab. We really want mnttab here anyways, so just use it.

# 1dc85563 16-Oct-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix destroy of imported BE w/ AUTOORIGIN

Imported BE, much like the activated BE, will not have an origin that we can
fetch/examine for destruction. be_destroy should not return BE_ERR_NOORIGIN
for failure to get the origin property for BE_DESTROY_AUTOORIGIN, because
we don't really know going into it that there's even an origin to be
destroyed.

BE_DESTROY_NEEDORIGIN has been renamed to BE_DESTROY_WANTORIGIN because only
a subset of it *needs* the origin, so 'need' is too strong of verbiage.

This was caught by jenkins and the bectl tests, but kevans failed to run the
bectl tests prior to commit.

Reported by: lwhsu

# 455d8009 16-Oct-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): add needed bits for be_destroy to auto-destroy some origins

New BEs can be created from either an existing snapshot or an existing BE.
If an existing BE is chosen (either implicitly via 'bectl create' or
explicitly via 'bectl create -e foo bar', for instance), then bectl will
create a snapshot of the current BE or "foo" with be_snapshot, with a name
formatted like: strftime("%F-%T") and a serial added to it.

This commit adds the needed bits for libbe or consumers to determine if a
snapshot names matches one of these auto-created snapshots (with some light
validation of the date/time/serial), and also a be_destroy flag to specify
that the origin should be automatically destroyed if possible.

A future commit to bectl will specify BE_DESTROY_AUTOORIGIN by default so we
clean up the origin in the most common case, non-user-managed snapshots.

# 8569a95e 17-Sep-2019 Andriy Gapon <avg@FreeBSD.org>

fixup up fallout from r352447 in libbe

I totally forgot that we now have another in-tree consumer of libzfs.

MFC after: 3 days
X-MFC with: r352447

# ac34fe23 02-May-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: set mountpoint=none in be_import

If we're going to set a mountpoint at all, mountpoint=none makes more sense
than mountpoint=/.

MFC after: 3 days

# be13d48c 25-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Copy received properties as well

This was inherently broken on send|recv datasets.

Reported and tested by: Wes Maag <jwmaag gmail com>
MFC after: 3 days

# fa30d9ed 22-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): allow creation of arbitrary depth boot environments

libbe currently only provides an API to create a recursive boot environment,
without any formal support for intentionally limiting the depth. This
changeset adds an API, be_create_depth, that may be used to arbitrarily
restrict the depth of the new BE.

Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18564

# fcb47c42 10-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): use libzfs name validation for datasets/snapshot names

Our home-rolled solution didn't quite capture all of the details, and we
didn't actually validate snapshot names at all. zfs_name_valid captures the
important details, but it doesn't necessarily expose the errors that we're
wanting to see in the be_validate_* functions. Validating lengths
independently, then the names, should make this a non-issue.

# 90cf61e8 03-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add a serial to the generated snapshot names

To use bectl in an example, when one creates a new boot environment with
either `bectl create <be>` or `bectl create -e <otherbe> <be>`, libbe will
take a snapshot of the original boot environment to clone. Previously, this
used %F-%T date format as the snapshot name, but this has some limitations-
attempting to create multiple boot environments in quick succession may
collide if done within the same second.

Tack a serial onto it to reduce the chances of a collision... we could still
collide if multiple processes/threads are creating boot environments at the
same time, but this is likely not a big concern as this has only been
reported as occurring in freebsd-ci setup.

MFC after: 3 days

# e1ee6230 01-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: Fix zfs_is_mounted check w/ snapshots

'be_destroy' can destroy a boot environment (by name) or a given snapshot.
If the target to be destroyed is a dataset, check if it's mounted. We don't
want to check if the origin dataset is mounted when destroying a snapshot.

PR: 236043
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19650

# be7dd423 12-Feb-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix be_destroy behavior w.r.t. deep BE snapshots and -o

be_destroy is documented to recursively destroy a boot environment. In the
case of snapshots, one would take this to mean that these are also
recursively destroyed. However, this was previously not the case.
be_destroy would descend into the be_destroy callback and attempt to
zfs_iter_children on the top-level snapshot, which is bogus.

Our alternative approach is to take note of the snapshot name and iterate
through all of fs children of the BE to try destruction in the children.

The -o option is also fixed to work properly with deep BEs. If the BE was
created with `bectl create -e otherDeepBE newDeepBE`, for instance, then a
recursive snapshot of otherDeepBE would have been taken for construction of
newDeepBE but a subsequent destroy with BE_DESTROY_ORIGIN set would only
clean up the snapshot at the root of otherDeepBE: ${BEROOT}/otherDeepBE@...

The most recent iteration instead pretends not to know how these things
work, verifies that the origin is another BE and then passes that back
through be_destroy to DTRT when snapshots and deep BEs may be in play.

MFC after: 1 week

# 13c62c50 10-Feb-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add a destroy option for removing the origin

Currently origin snapshots are left behind when a BE is destroyed, whether
it was an auto-created snapshot or explicitly specified via, for example,
`bectl create -e be@mysnap ...`.

Removing it automatically could be argued as a POLA violation in some
circumstances, so provide a flag to be_destroy for it. An accompanying
option will be added to bectl(8) to utilize this.

Some minor style/consistency nits in the affected areas also addressed.

Reported by: Shawn Webb
MFC after: 1 week

# 16ac0705 22-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): simplify import, allow replication streams

Previously, we directly used libzfs_core's lzc_receive to import to a
temporary snapshot, then cloned the snapshot and setup the properties. This
failed when attempting to import replication streams with questionable
error.

libzfs's zfs_receive is a much better fit here, so we now use it instead
with the destination dataset and let libzfs take care of the dirty details.
be_import is greatly simplified as a result.

Reported by: Marie Helene Kvello-Aune <freebsd@mhka.no>
MFC after: 1 week

# fc13fc1c 09-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): move altroot augmentation bits around a little bit

We could perhaps have a method that does this given a dataset, but it's yet
clear that we'll always want to bypass the altroot when we grab the
mountpoint. For now, we'll refactor things a bit so we grab the altroot
length when libbe is initialized and have a common method that does the
necessary augmentation (replace with / if it's the root, return a pointer to
later in the string if not).

This will be used in some upcoming work to make be_mount work properly for
deep BEs.

MFC after: 1 week

# f08dac4e 07-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Don't allow bootfs to be destroyed

Previously, the following sequence of events was feasible under some
circumstance:

bectl create test
bectl activate test
# the test BE dataset gets promoted and set as bootfs
bectl destroy test

I was unable to reproduce the destroy succeeding, but we should be rejecting
this before it even gets to libzfs because it would leave the system in an
inconsistent state. Forcing the user to be explicit as to which environment
should be activated instead is much better.

Reported by: Graham Perrin <grahamperrin@gmail.com>
MFC after: 3 days

# 4ab5187d 19-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Handle non-ZFS rootfs better

If rootfs isn't ZFS, current version will emit an error claiming so and fail
to initialize libbe. As a consumer, bectl -r (undocumented) can be specified
to operate on a BE independently of whether on a UFS or ZFS root.

Unbreak this for the UFS case by only erroring out the init if we can't
determine a ZFS dataset for rootfs and no BE root was specified. Consumers
of libbe should take care to ensure that rootfs is non-empty if they're
trying to use it, because this could certainly be the case.

Some check is needed before zfs_path_to_zhandle because it will
unconditionally emit to stderr if the path isn't a ZFS filesystem, which is
unhelpful for our purposes.

This should also unbreak the bectl(8) tests on a UFS root, as is the case in
Jenkins' -test runs.

MFC after: 3 days

# af43c24d 18-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Properly account for altroot when creating new BEs

Previously we would blindly copy the 'mountpoint' property, which includes
the altroot. The altroot needs to be snipped off prior to setting it on the
new BE, though, or you'll end up with a new BE and a mountpoint of /mnt with
altroot=/mnt

MFC after: 3 days

# cc624025 18-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

bectl(3)/libbe(3): Allow BE root to be specified

Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:

- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)

Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.

bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt

this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.

MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029

# 51aecc89 17-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): rewrite init to support chroot usage

libbe(3) currently uses zfs_be_root and locates which of its children is
currently mounted at "/". This is reasonable, but not correct in the case of
a chroot, for two reasons:

- chroot root may be of a different zpool than zfs_be_root
- chroot root will not show up as mounted at "/"

Fix both of these by rewriting libbe_init to work from the rootfs down.
zfs_path_to_zhandle on / will resolve to the dataset mounted at the new
root, rather than the real root. From there, we can derive the BE root/pool
and grab the bootfs off of the new pool. This does no harm in the average
case, and opens up bectl to operating on different pools for scenarios where
one may be, for instance, updating a pool that generally gets re-rooted into
from a separate UFS root or zfs bootpool.

While here, I've also:
- Eliminated the check for /boot and / to be on the same partition. This
leaves one open to a setup where /boot (and consequently, kernel/modules)
are not included in the boot environment. This may very well be an
intentional setup done by someone that knows what they're doing, we should
not kill BE usage because of it.

- Eliminated the validation bits of BEs and snapshots that enforced
'mountpoint' to be "/" -- this broke when trying to operate on an imported
pool with an altroot, but we need not be this picky.

Reported by: philip
Reviewed by: philip, allanjude (previous version)
Tested by: philip
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18012

# 8d4ce358 10-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Set canmount properly when activating a new BE

The previously activated BE should have canmount=noauto set on it upon
activation of the new BE, but we previously did not touch canmount on either
old or new BE.

PR: 233113
MFC after: 3 days

# 4635676d 01-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Don't promote non-cloned BEs

Most easily reproducible by attempting to activate the currently activated
BE, one would get a "not a cloned filesystem" error instead of success or a
sane message.

PR: 232488
MFC after: 3 days

# 0cadc427 01-Oct-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix BE activation promoting activated BE

This allows older BEs to be destroyed as they become replaced by a BE
created from them: e.g.

bectl create -e brokenworld fixedworld
bectl activate fixedworld
bectl destroy brokenworld

Submitted by: Shawn Webb
Approved by: re (gjb)
Obtained from: HardenedBSD (5948c0581e)

# 162ec569 31-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix error handling with respect to be_exists

Some paths through be_exists will set the error state, others will not
There are multiple reasons that a call can fail, so clean it up a bit: all
paths now return an appropriate error code so the caller can attempt to
distinguish between a BE legitimately not existing and just having the wrong
mountpoint. The caller is expected to bubble the error through to the
internal error handler as needed.

This fixes some unfriendliness with bectl(8)'s activate subcommand, where
it might fail due to a bad mountpoint but the only message output is a
generic "failed to activate" message.

Approved by: re (gjb)

# 8369ba42 24-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Make consistent with beadm

vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.

Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.

A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.

Approved by: re (kib)

# 5b7803a9 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Impose dataset length restrictions on boot env name validation

Previously, we only validated names for character restrictions. This is
helpful, but we should've also checked length restrictions- dataset names
must be restricted to MAXNAMELEN.

While here, move validation before doing a bunch of concatenations and fix
error handling in be_rename. It was previously setting the error state based
on return value from a libzfs function, which is wrong: libzfs errors don't
necessarily match cleanly to libbe errors. This would cause the assertion in
be_error to hit when the error was printed.

# a8e44f4d 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Prefer safer versions of strcat/strcpy

Or, in the activate case, just use snprintf since that's effectively what
we're doing anyways.

# 55b0e92b 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Hit rewind on a bunch of off-by-ones

While here, use sizeof() in some places that it makes sense to reduce room
for error and prefer strlcpy to strncpy

# cc4deabc 14-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix leaky faucets

Amongst them:
- Resource leaks
- Logically dead code
- Unused values
- Null termination issues

Reported by: asomers (pointer to Coverity), Coverity
CID: 1394777, 1394791, 1394830, 1394844, 1394872, 1394894,
CID: 1394900, 1394907, 1394950, 1394965

# 00ece7c2 12-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Light typo fix/word addition

# 1b057aac 12-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix be_import to delete temp snapshot

Deleting the temp snapshot isn't immediately possible because it's the
origin of the newly imported boot environment. However, this is trivially
solved by opening the new boot environment and promoting it. The roles are
now reversed and the temp snapshot/dataset may be completely destroyed.

Remove the BUGS from libbe(3) and bectl(8).

# d2a6bc9f 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Document the import bug...

# 3d1a1f2c 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Kill off the 'add' functionality for now

The mostly-undocumented 'add' functionality, from initial read-through, is
intended for construction of deep ("bdrewery style") boot environments.
However, it's mostly broken at this point. `#if SOON` it out on both sides
so that we're not exposing a broken API/feature.

Work will resume on it in due time.

# c65a2111 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): More error handling bits

be_add_child functionality gets split out into separate places as a bonus.
A lot of places here we'll gloss over libzfs errors, because they shouldn't
be happening given the conditions that we're operating under. "Unknown
error" is what I'm intending to use for the moment to indicate an
exceptional circumstance- exceptional enough that we can't tell the consumer
did because we're not so certain that they did anything.

# 6d4b1d24 09-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Plug some holes, do some more proper error returns

For those returning just -1 before, have them set ERR_UNKNOWN for now.

# 73c3d608 09-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): more small cleanup, const'ify and light style(9)

# 506f5fdf 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Some more light error handling...

# 2989df09 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Clarify some errors

While here, fix a bug with 'rename' that checked the wrong name for being
the active BE.

# b6e7c421 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Standardize $FreeBSD$ IDs

# b179da01 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Standardize copyright headers

- File names don't necessarily need to be repeated
- Add SPDX tags
- Add a missing copyright for Kyle Kneitinger in bectl.8, originally written
by him in GSoC 2017; his standard copyright notice has been copied from
other files within the same directory to remain consistent with how he
clearly wished to portray it

# 920abf4d 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Destroy all children of a BE dataset, too

This fixes destruction of a deep BE returning an EBUSY because child
datasets still exist.

# 3f4e6c96 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

This snippet is no longer from zfsbootcfg

# d06f7103 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Rewrite activate temp bits to rely less on loader

Loader is still relied upon at the beginning of libbe to specify the be
root, but we can derive from that the primary zpool and any vdevs that we
need to set nextboot bits on.

This lets me successfully `bectl activate -t test`, but UEFI loader doesn't
quite yet understand so it's effectively defunct.

# 9b1662e6 04-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

bectl: Implement -D ("space if origin datasets were deleted")

This also accomplishes the following:

- Proxy through zfs_nicenum as be_nicenum, because it looks better than
humanize_number and would presumably be useful to other libbe consumers.

- Rename be_get_snapshot_props to be_get_dataset_props, make it more useful

# b29bf2f8 25-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/be(8): Drop WARNS overrides, fix all fallout

Based on the idea that we shouldn't have all-new library and utility going
into base that need WARNS=1...

- Decent amount of constification
- Lots of parentheses
- Minor other nits

# ff8676cc 25-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add nextboot flag to returned BE information

# bfe0869c 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): make style consistent with what I'll use going forward

# ee16b7c9 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Find rootfs instead by enumerating child datasets of BE root

This makes us more resilient to a rename of the bootfs, but still wouldn't
withstand pool renames or guid renames. More importantly, this allows `bectl
create <foo>` work out of the box to create a boot environment based on the
currently booted one.

# c3a34c08 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Disambiguate 'active' a little bit, add 'bootfs'

- Rename 'active' to 'rootfs', which is used in other places to describe the
currently booted (or about to be booted) BE.

- Add 'bootfs', which indicates the next boot environment to be booted. This
is pulled from the BOOTFS zpool property.

- Go ahead and keep an open handle to the active zpool. We might need to
enumerate datasets, get properties, and set properties (e.g. bootfs)
throughout other libbe bits, and a single handle isn't overly expensive.

# 28f16a0f 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

Import libbe(3)/be(1) from socsvn/soc2017/kneitinger/libbe-head

# 8e933d9c 14-Mar-2023 John Grafton <john.grafton@gmail.com>

libbe: Avoid double printing cloning errors.

be_clone calls be_clone_cb and both call set_error on the return
error path. set_error prints the error resulting in a double print.
be_clone_cb should just return the error code and allow be_clone
to print it.

PR: 265248
Reported by: Graham Perrin
Reviewed by: imp, kevans
Pull Request: https://github.com/freebsd/freebsd-src/pull/685


# 15f0b8c3 25-Jan-2023 Martin Matuska <mm@FreeBSD.org>

zfs: merge openzfs/zfs@9cd71c860 (master)

Notable upstream pull request merges:
#13805 Configure zed's diagnosis engine with vdev properties
#14110 zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
#14121 Batch enqueue/dequeue for bqueue
#14123 arc_read()/arc_access() refactoring and cleanup
#14159 Bypass metaslab throttle for removal allocations
#14243 Implement uncached prefetch
#14251 Cache dbuf_hash() calculation
#14253 Allow reciever to override encryption property in case of replication
#14254 Restrict visibility of per-dataset kstats inside FreeBSD jails
#14255 Zero end of embedded block buffer in dump_write_embedded()
#14263 Cleanups identified by CodeQL and Coverity
#14264 Miscellaneous fixes
#14272 Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names
#14287 FreeBSD: Remove stray debug printf
#14288 Colorize zfs diff output
#14289 deadlock between spa_errlog_lock and dp_config_rwlock
#14291 FreeBSD: Fix potential boot panic with bad label
#14292 Add tunable to allow changing micro ZAP's max size
#14293 Turn default_bs and default_ibs into ZFS_MODULE_PARAMs
#14295 zed: add hotplug support for spare vdevs
#14304 Activate filesystem features only in syncing context
#14311 zpool: do guid-based comparison in is_vdev_cb()
#14317 Pack zrlock_t by 8 bytes
#14320 Update arc_summary and arcstat outputs
#14328 FreeBSD: catch up to 1400077
#14376 Use setproctitle to report progress of zfs send
#14340 Remove some dead ARC code
#14358 Wait for txg sync if the last DRR_FREEOBJECTS might result in a hole
#14360 libzpool: fix ddi_strtoull to update nptr
#14364 Fix unprotected zfs_znode_dmu_fini
#14379 zfs_receive_one: Check for the more likely error first
#14380 Cleanup of dead code suggested by Clang Static Analyzer
#14397 Avoid passing an uninitialized index to dsl_prop_known_index
#14404 Fix reading uninitialized variable in receive_read
#14407 free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report
#14418 Introduce minimal ZIL block commit delay
#14422 x86 assembly: fix .size placement and replace .align with .balign

Obtained from: OpenZFS
OpenZFS commit: 9cd71c8604d52def22ffaddc35755712f0fb9349


# dadb9c70 02-Apr-2022 Kyle Evans <kevans@FreeBSD.org>

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.

While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.

Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605

# 7edc1bd9 08-Jan-2021 Gleb Smirnoff <glebius@FreeBSD.org>

When be_activate() turns on a new boot environment, it always deactivates
the current one first. And if it fails to do so, it abandons activation.
However, with the new bootonce feature, there is a legitimate case when
a pool doesn't have "bootfs" property set. Check for this case before
calling be_deactivate().

Reviewed by: kevans

# e307eb94 21-Sep-2020 Toomas Soome <tsoome@FreeBSD.org>

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

# eac7052f 04-Sep-2020 Matt Macy <mmacy@FreeBSD.org>

ZFS: MFV 2.0-rc1-gfd20a8

- fixes jail delegation
- fixes raw kstat unsupported complaints
- exposes dbgmsg, dbuf, and checksum benchmark stats
- restore rename -u support


# 9e5787d2 24-Aug-2020 Matt Macy <mmacy@FreeBSD.org>

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

# 9d6d8bf8 21-Jul-2020 Kyle Evans <kevans@FreeBSD.org>

libbe: annotate lbh as __unused in be_is_auto_snapshot_name

lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a post-OpenZFS-import world will be able to raise WARNS of
libbe to the default (pending some minor changes to openzfs libzfs).

MFC after: 3 days

# 6966ac05 23-Jan-2020 Kyle Evans <kevans@FreeBSD.org>

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.

# 8f5c6c31 02-Jan-2020 Kyle Evans <kevans@FreeBSD.org>

libbe(3): promote dependent clones when destroying an environment

When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.

This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.

PR: 242592
Submitted by: Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22953

# 485172f5 06-Dec-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: fix build against sysutils/openzfs, part 1

This is the half of the changes required that work as-is with both in-tree
ZFS and the new hotness, sysutils/openzfs. Highlights are less dependency
on header pollution (from somewhere) and using 'mnttab' instead of
'extmnttab'. In the in-tree ZFS, the latter is a #define for the former,
but in the port extmnttab is actually a distinct struct that's a super-set
of mnttab. We really want mnttab here anyways, so just use it.

# 1dc85563 16-Oct-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix destroy of imported BE w/ AUTOORIGIN

Imported BE, much like the activated BE, will not have an origin that we can
fetch/examine for destruction. be_destroy should not return BE_ERR_NOORIGIN
for failure to get the origin property for BE_DESTROY_AUTOORIGIN, because
we don't really know going into it that there's even an origin to be
destroyed.

BE_DESTROY_NEEDORIGIN has been renamed to BE_DESTROY_WANTORIGIN because only
a subset of it *needs* the origin, so 'need' is too strong of verbiage.

This was caught by jenkins and the bectl tests, but kevans failed to run the
bectl tests prior to commit.

Reported by: lwhsu

# 455d8009 16-Oct-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): add needed bits for be_destroy to auto-destroy some origins

New BEs can be created from either an existing snapshot or an existing BE.
If an existing BE is chosen (either implicitly via 'bectl create' or
explicitly via 'bectl create -e foo bar', for instance), then bectl will
create a snapshot of the current BE or "foo" with be_snapshot, with a name
formatted like: strftime("%F-%T") and a serial added to it.

This commit adds the needed bits for libbe or consumers to determine if a
snapshot names matches one of these auto-created snapshots (with some light
validation of the date/time/serial), and also a be_destroy flag to specify
that the origin should be automatically destroyed if possible.

A future commit to bectl will specify BE_DESTROY_AUTOORIGIN by default so we
clean up the origin in the most common case, non-user-managed snapshots.

# 8569a95e 17-Sep-2019 Andriy Gapon <avg@FreeBSD.org>

fixup up fallout from r352447 in libbe

I totally forgot that we now have another in-tree consumer of libzfs.

MFC after: 3 days
X-MFC with: r352447

# ac34fe23 02-May-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: set mountpoint=none in be_import

If we're going to set a mountpoint at all, mountpoint=none makes more sense
than mountpoint=/.

MFC after: 3 days

# be13d48c 25-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Copy received properties as well

This was inherently broken on send|recv datasets.

Reported and tested by: Wes Maag <jwmaag gmail com>
MFC after: 3 days

# fa30d9ed 22-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): allow creation of arbitrary depth boot environments

libbe currently only provides an API to create a recursive boot environment,
without any formal support for intentionally limiting the depth. This
changeset adds an API, be_create_depth, that may be used to arbitrarily
restrict the depth of the new BE.

Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18564

# fcb47c42 10-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): use libzfs name validation for datasets/snapshot names

Our home-rolled solution didn't quite capture all of the details, and we
didn't actually validate snapshot names at all. zfs_name_valid captures the
important details, but it doesn't necessarily expose the errors that we're
wanting to see in the be_validate_* functions. Validating lengths
independently, then the names, should make this a non-issue.

# 90cf61e8 03-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add a serial to the generated snapshot names

To use bectl in an example, when one creates a new boot environment with
either `bectl create <be>` or `bectl create -e <otherbe> <be>`, libbe will
take a snapshot of the original boot environment to clone. Previously, this
used %F-%T date format as the snapshot name, but this has some limitations-
attempting to create multiple boot environments in quick succession may
collide if done within the same second.

Tack a serial onto it to reduce the chances of a collision... we could still
collide if multiple processes/threads are creating boot environments at the
same time, but this is likely not a big concern as this has only been
reported as occurring in freebsd-ci setup.

MFC after: 3 days

# e1ee6230 01-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: Fix zfs_is_mounted check w/ snapshots

'be_destroy' can destroy a boot environment (by name) or a given snapshot.
If the target to be destroyed is a dataset, check if it's mounted. We don't
want to check if the origin dataset is mounted when destroying a snapshot.

PR: 236043
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19650

# be7dd423 12-Feb-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix be_destroy behavior w.r.t. deep BE snapshots and -o

be_destroy is documented to recursively destroy a boot environment. In the
case of snapshots, one would take this to mean that these are also
recursively destroyed. However, this was previously not the case.
be_destroy would descend into the be_destroy callback and attempt to
zfs_iter_children on the top-level snapshot, which is bogus.

Our alternative approach is to take note of the snapshot name and iterate
through all of fs children of the BE to try destruction in the children.

The -o option is also fixed to work properly with deep BEs. If the BE was
created with `bectl create -e otherDeepBE newDeepBE`, for instance, then a
recursive snapshot of otherDeepBE would have been taken for construction of
newDeepBE but a subsequent destroy with BE_DESTROY_ORIGIN set would only
clean up the snapshot at the root of otherDeepBE: ${BEROOT}/otherDeepBE@...

The most recent iteration instead pretends not to know how these things
work, verifies that the origin is another BE and then passes that back
through be_destroy to DTRT when snapshots and deep BEs may be in play.

MFC after: 1 week

# 13c62c50 10-Feb-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add a destroy option for removing the origin

Currently origin snapshots are left behind when a BE is destroyed, whether
it was an auto-created snapshot or explicitly specified via, for example,
`bectl create -e be@mysnap ...`.

Removing it automatically could be argued as a POLA violation in some
circumstances, so provide a flag to be_destroy for it. An accompanying
option will be added to bectl(8) to utilize this.

Some minor style/consistency nits in the affected areas also addressed.

Reported by: Shawn Webb
MFC after: 1 week

# 16ac0705 22-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): simplify import, allow replication streams

Previously, we directly used libzfs_core's lzc_receive to import to a
temporary snapshot, then cloned the snapshot and setup the properties. This
failed when attempting to import replication streams with questionable
error.

libzfs's zfs_receive is a much better fit here, so we now use it instead
with the destination dataset and let libzfs take care of the dirty details.
be_import is greatly simplified as a result.

Reported by: Marie Helene Kvello-Aune <freebsd@mhka.no>
MFC after: 1 week

# fc13fc1c 09-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): move altroot augmentation bits around a little bit

We could perhaps have a method that does this given a dataset, but it's yet
clear that we'll always want to bypass the altroot when we grab the
mountpoint. For now, we'll refactor things a bit so we grab the altroot
length when libbe is initialized and have a common method that does the
necessary augmentation (replace with / if it's the root, return a pointer to
later in the string if not).

This will be used in some upcoming work to make be_mount work properly for
deep BEs.

MFC after: 1 week

# f08dac4e 07-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Don't allow bootfs to be destroyed

Previously, the following sequence of events was feasible under some
circumstance:

bectl create test
bectl activate test
# the test BE dataset gets promoted and set as bootfs
bectl destroy test

I was unable to reproduce the destroy succeeding, but we should be rejecting
this before it even gets to libzfs because it would leave the system in an
inconsistent state. Forcing the user to be explicit as to which environment
should be activated instead is much better.

Reported by: Graham Perrin <grahamperrin@gmail.com>
MFC after: 3 days

# 4ab5187d 19-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Handle non-ZFS rootfs better

If rootfs isn't ZFS, current version will emit an error claiming so and fail
to initialize libbe. As a consumer, bectl -r (undocumented) can be specified
to operate on a BE independently of whether on a UFS or ZFS root.

Unbreak this for the UFS case by only erroring out the init if we can't
determine a ZFS dataset for rootfs and no BE root was specified. Consumers
of libbe should take care to ensure that rootfs is non-empty if they're
trying to use it, because this could certainly be the case.

Some check is needed before zfs_path_to_zhandle because it will
unconditionally emit to stderr if the path isn't a ZFS filesystem, which is
unhelpful for our purposes.

This should also unbreak the bectl(8) tests on a UFS root, as is the case in
Jenkins' -test runs.

MFC after: 3 days

# af43c24d 18-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Properly account for altroot when creating new BEs

Previously we would blindly copy the 'mountpoint' property, which includes
the altroot. The altroot needs to be snipped off prior to setting it on the
new BE, though, or you'll end up with a new BE and a mountpoint of /mnt with
altroot=/mnt

MFC after: 3 days

# cc624025 18-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

bectl(3)/libbe(3): Allow BE root to be specified

Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:

- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)

Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.

bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt

this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.

MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029

# 51aecc89 17-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): rewrite init to support chroot usage

libbe(3) currently uses zfs_be_root and locates which of its children is
currently mounted at "/". This is reasonable, but not correct in the case of
a chroot, for two reasons:

- chroot root may be of a different zpool than zfs_be_root
- chroot root will not show up as mounted at "/"

Fix both of these by rewriting libbe_init to work from the rootfs down.
zfs_path_to_zhandle on / will resolve to the dataset mounted at the new
root, rather than the real root. From there, we can derive the BE root/pool
and grab the bootfs off of the new pool. This does no harm in the average
case, and opens up bectl to operating on different pools for scenarios where
one may be, for instance, updating a pool that generally gets re-rooted into
from a separate UFS root or zfs bootpool.

While here, I've also:
- Eliminated the check for /boot and / to be on the same partition. This
leaves one open to a setup where /boot (and consequently, kernel/modules)
are not included in the boot environment. This may very well be an
intentional setup done by someone that knows what they're doing, we should
not kill BE usage because of it.

- Eliminated the validation bits of BEs and snapshots that enforced
'mountpoint' to be "/" -- this broke when trying to operate on an imported
pool with an altroot, but we need not be this picky.

Reported by: philip
Reviewed by: philip, allanjude (previous version)
Tested by: philip
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18012

# 8d4ce358 10-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Set canmount properly when activating a new BE

The previously activated BE should have canmount=noauto set on it upon
activation of the new BE, but we previously did not touch canmount on either
old or new BE.

PR: 233113
MFC after: 3 days

# 4635676d 01-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Don't promote non-cloned BEs

Most easily reproducible by attempting to activate the currently activated
BE, one would get a "not a cloned filesystem" error instead of success or a
sane message.

PR: 232488
MFC after: 3 days

# 0cadc427 01-Oct-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix BE activation promoting activated BE

This allows older BEs to be destroyed as they become replaced by a BE
created from them: e.g.

bectl create -e brokenworld fixedworld
bectl activate fixedworld
bectl destroy brokenworld

Submitted by: Shawn Webb
Approved by: re (gjb)
Obtained from: HardenedBSD (5948c0581e)

# 162ec569 31-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix error handling with respect to be_exists

Some paths through be_exists will set the error state, others will not
There are multiple reasons that a call can fail, so clean it up a bit: all
paths now return an appropriate error code so the caller can attempt to
distinguish between a BE legitimately not existing and just having the wrong
mountpoint. The caller is expected to bubble the error through to the
internal error handler as needed.

This fixes some unfriendliness with bectl(8)'s activate subcommand, where
it might fail due to a bad mountpoint but the only message output is a
generic "failed to activate" message.

Approved by: re (gjb)

# 8369ba42 24-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Make consistent with beadm

vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.

Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.

A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.

Approved by: re (kib)

# 5b7803a9 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Impose dataset length restrictions on boot env name validation

Previously, we only validated names for character restrictions. This is
helpful, but we should've also checked length restrictions- dataset names
must be restricted to MAXNAMELEN.

While here, move validation before doing a bunch of concatenations and fix
error handling in be_rename. It was previously setting the error state based
on return value from a libzfs function, which is wrong: libzfs errors don't
necessarily match cleanly to libbe errors. This would cause the assertion in
be_error to hit when the error was printed.

# a8e44f4d 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Prefer safer versions of strcat/strcpy

Or, in the activate case, just use snprintf since that's effectively what
we're doing anyways.

# 55b0e92b 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Hit rewind on a bunch of off-by-ones

While here, use sizeof() in some places that it makes sense to reduce room
for error and prefer strlcpy to strncpy

# cc4deabc 14-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix leaky faucets

Amongst them:
- Resource leaks
- Logically dead code
- Unused values
- Null termination issues

Reported by: asomers (pointer to Coverity), Coverity
CID: 1394777, 1394791, 1394830, 1394844, 1394872, 1394894,
CID: 1394900, 1394907, 1394950, 1394965

# 00ece7c2 12-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Light typo fix/word addition

# 1b057aac 12-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix be_import to delete temp snapshot

Deleting the temp snapshot isn't immediately possible because it's the
origin of the newly imported boot environment. However, this is trivially
solved by opening the new boot environment and promoting it. The roles are
now reversed and the temp snapshot/dataset may be completely destroyed.

Remove the BUGS from libbe(3) and bectl(8).

# d2a6bc9f 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Document the import bug...

# 3d1a1f2c 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Kill off the 'add' functionality for now

The mostly-undocumented 'add' functionality, from initial read-through, is
intended for construction of deep ("bdrewery style") boot environments.
However, it's mostly broken at this point. `#if SOON` it out on both sides
so that we're not exposing a broken API/feature.

Work will resume on it in due time.

# c65a2111 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): More error handling bits

be_add_child functionality gets split out into separate places as a bonus.
A lot of places here we'll gloss over libzfs errors, because they shouldn't
be happening given the conditions that we're operating under. "Unknown
error" is what I'm intending to use for the moment to indicate an
exceptional circumstance- exceptional enough that we can't tell the consumer
did because we're not so certain that they did anything.

# 6d4b1d24 09-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Plug some holes, do some more proper error returns

For those returning just -1 before, have them set ERR_UNKNOWN for now.

# 73c3d608 09-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): more small cleanup, const'ify and light style(9)

# 506f5fdf 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Some more light error handling...

# 2989df09 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Clarify some errors

While here, fix a bug with 'rename' that checked the wrong name for being
the active BE.

# b6e7c421 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Standardize $FreeBSD$ IDs

# b179da01 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Standardize copyright headers

- File names don't necessarily need to be repeated
- Add SPDX tags
- Add a missing copyright for Kyle Kneitinger in bectl.8, originally written
by him in GSoC 2017; his standard copyright notice has been copied from
other files within the same directory to remain consistent with how he
clearly wished to portray it

# 920abf4d 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Destroy all children of a BE dataset, too

This fixes destruction of a deep BE returning an EBUSY because child
datasets still exist.

# 3f4e6c96 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

This snippet is no longer from zfsbootcfg

# d06f7103 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Rewrite activate temp bits to rely less on loader

Loader is still relied upon at the beginning of libbe to specify the be
root, but we can derive from that the primary zpool and any vdevs that we
need to set nextboot bits on.

This lets me successfully `bectl activate -t test`, but UEFI loader doesn't
quite yet understand so it's effectively defunct.

# 9b1662e6 04-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

bectl: Implement -D ("space if origin datasets were deleted")

This also accomplishes the following:

- Proxy through zfs_nicenum as be_nicenum, because it looks better than
humanize_number and would presumably be useful to other libbe consumers.

- Rename be_get_snapshot_props to be_get_dataset_props, make it more useful

# b29bf2f8 25-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/be(8): Drop WARNS overrides, fix all fallout

Based on the idea that we shouldn't have all-new library and utility going
into base that need WARNS=1...

- Decent amount of constification
- Lots of parentheses
- Minor other nits

# ff8676cc 25-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add nextboot flag to returned BE information

# bfe0869c 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): make style consistent with what I'll use going forward

# ee16b7c9 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Find rootfs instead by enumerating child datasets of BE root

This makes us more resilient to a rename of the bootfs, but still wouldn't
withstand pool renames or guid renames. More importantly, this allows `bectl
create <foo>` work out of the box to create a boot environment based on the
currently booted one.

# c3a34c08 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Disambiguate 'active' a little bit, add 'bootfs'

- Rename 'active' to 'rootfs', which is used in other places to describe the
currently booted (or about to be booted) BE.

- Add 'bootfs', which indicates the next boot environment to be booted. This
is pulled from the BOOTFS zpool property.

- Go ahead and keep an open handle to the active zpool. We might need to
enumerate datasets, get properties, and set properties (e.g. bootfs)
throughout other libbe bits, and a single handle isn't overly expensive.

# 28f16a0f 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

Import libbe(3)/be(1) from socsvn/soc2017/kneitinger/libbe-head

# dadb9c70 02-Apr-2022 Kyle Evans <kevans@FreeBSD.org>

bectl: push space-in-name check down into libbe

This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.

While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.

Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605


# 7edc1bd9 08-Jan-2021 Gleb Smirnoff <glebius@FreeBSD.org>

When be_activate() turns on a new boot environment, it always deactivates
the current one first. And if it fails to do so, it abandons activation.
However, with the new bootonce feature, there is a legitimate case when
a pool doesn't have "bootfs" property set. Check for this case before
calling be_deactivate().

Reviewed by: kevans


# e307eb94 21-Sep-2020 Toomas Soome <tsoome@FreeBSD.org>

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512


# 9e5787d2 24-Aug-2020 Matt Macy <mmacy@FreeBSD.org>

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872


# 9d6d8bf8 21-Jul-2020 Kyle Evans <kevans@FreeBSD.org>

libbe: annotate lbh as __unused in be_is_auto_snapshot_name

lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a post-OpenZFS-import world will be able to raise WARNS of
libbe to the default (pending some minor changes to openzfs libzfs).

MFC after: 3 days


# 6966ac05 23-Jan-2020 Kyle Evans <kevans@FreeBSD.org>

Drop "All Rights Reserved" from all libbe/bectl files

I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.


# 8f5c6c31 02-Jan-2020 Kyle Evans <kevans@FreeBSD.org>

libbe(3): promote dependent clones when destroying an environment

When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.

This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.

PR: 242592
Submitted by: Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22953


# 485172f5 06-Dec-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: fix build against sysutils/openzfs, part 1

This is the half of the changes required that work as-is with both in-tree
ZFS and the new hotness, sysutils/openzfs. Highlights are less dependency
on header pollution (from somewhere) and using 'mnttab' instead of
'extmnttab'. In the in-tree ZFS, the latter is a #define for the former,
but in the port extmnttab is actually a distinct struct that's a super-set
of mnttab. We really want mnttab here anyways, so just use it.


# 1dc85563 16-Oct-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix destroy of imported BE w/ AUTOORIGIN

Imported BE, much like the activated BE, will not have an origin that we can
fetch/examine for destruction. be_destroy should not return BE_ERR_NOORIGIN
for failure to get the origin property for BE_DESTROY_AUTOORIGIN, because
we don't really know going into it that there's even an origin to be
destroyed.

BE_DESTROY_NEEDORIGIN has been renamed to BE_DESTROY_WANTORIGIN because only
a subset of it *needs* the origin, so 'need' is too strong of verbiage.

This was caught by jenkins and the bectl tests, but kevans failed to run the
bectl tests prior to commit.

Reported by: lwhsu


# 455d8009 16-Oct-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): add needed bits for be_destroy to auto-destroy some origins

New BEs can be created from either an existing snapshot or an existing BE.
If an existing BE is chosen (either implicitly via 'bectl create' or
explicitly via 'bectl create -e foo bar', for instance), then bectl will
create a snapshot of the current BE or "foo" with be_snapshot, with a name
formatted like: strftime("%F-%T") and a serial added to it.

This commit adds the needed bits for libbe or consumers to determine if a
snapshot names matches one of these auto-created snapshots (with some light
validation of the date/time/serial), and also a be_destroy flag to specify
that the origin should be automatically destroyed if possible.

A future commit to bectl will specify BE_DESTROY_AUTOORIGIN by default so we
clean up the origin in the most common case, non-user-managed snapshots.


# 8569a95e 17-Sep-2019 Andriy Gapon <avg@FreeBSD.org>

fixup up fallout from r352447 in libbe

I totally forgot that we now have another in-tree consumer of libzfs.

MFC after: 3 days
X-MFC with: r352447


# ac34fe23 02-May-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: set mountpoint=none in be_import

If we're going to set a mountpoint at all, mountpoint=none makes more sense
than mountpoint=/.

MFC after: 3 days


# be13d48c 25-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Copy received properties as well

This was inherently broken on send|recv datasets.

Reported and tested by: Wes Maag <jwmaag gmail com>
MFC after: 3 days


# fa30d9ed 22-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): allow creation of arbitrary depth boot environments

libbe currently only provides an API to create a recursive boot environment,
without any formal support for intentionally limiting the depth. This
changeset adds an API, be_create_depth, that may be used to arbitrarily
restrict the depth of the new BE.

Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18564


# fcb47c42 10-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): use libzfs name validation for datasets/snapshot names

Our home-rolled solution didn't quite capture all of the details, and we
didn't actually validate snapshot names at all. zfs_name_valid captures the
important details, but it doesn't necessarily expose the errors that we're
wanting to see in the be_validate_* functions. Validating lengths
independently, then the names, should make this a non-issue.


# 90cf61e8 03-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add a serial to the generated snapshot names

To use bectl in an example, when one creates a new boot environment with
either `bectl create <be>` or `bectl create -e <otherbe> <be>`, libbe will
take a snapshot of the original boot environment to clone. Previously, this
used %F-%T date format as the snapshot name, but this has some limitations-
attempting to create multiple boot environments in quick succession may
collide if done within the same second.

Tack a serial onto it to reduce the chances of a collision... we could still
collide if multiple processes/threads are creating boot environments at the
same time, but this is likely not a big concern as this has only been
reported as occurring in freebsd-ci setup.

MFC after: 3 days


# e1ee6230 01-Apr-2019 Kyle Evans <kevans@FreeBSD.org>

libbe: Fix zfs_is_mounted check w/ snapshots

'be_destroy' can destroy a boot environment (by name) or a given snapshot.
If the target to be destroyed is a dataset, check if it's mounted. We don't
want to check if the origin dataset is mounted when destroying a snapshot.

PR: 236043
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19650


# be7dd423 12-Feb-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix be_destroy behavior w.r.t. deep BE snapshots and -o

be_destroy is documented to recursively destroy a boot environment. In the
case of snapshots, one would take this to mean that these are also
recursively destroyed. However, this was previously not the case.
be_destroy would descend into the be_destroy callback and attempt to
zfs_iter_children on the top-level snapshot, which is bogus.

Our alternative approach is to take note of the snapshot name and iterate
through all of fs children of the BE to try destruction in the children.

The -o option is also fixed to work properly with deep BEs. If the BE was
created with `bectl create -e otherDeepBE newDeepBE`, for instance, then a
recursive snapshot of otherDeepBE would have been taken for construction of
newDeepBE but a subsequent destroy with BE_DESTROY_ORIGIN set would only
clean up the snapshot at the root of otherDeepBE: ${BEROOT}/otherDeepBE@...

The most recent iteration instead pretends not to know how these things
work, verifies that the origin is another BE and then passes that back
through be_destroy to DTRT when snapshots and deep BEs may be in play.

MFC after: 1 week


# 13c62c50 10-Feb-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add a destroy option for removing the origin

Currently origin snapshots are left behind when a BE is destroyed, whether
it was an auto-created snapshot or explicitly specified via, for example,
`bectl create -e be@mysnap ...`.

Removing it automatically could be argued as a POLA violation in some
circumstances, so provide a flag to be_destroy for it. An accompanying
option will be added to bectl(8) to utilize this.

Some minor style/consistency nits in the affected areas also addressed.

Reported by: Shawn Webb
MFC after: 1 week


# 16ac0705 22-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): simplify import, allow replication streams

Previously, we directly used libzfs_core's lzc_receive to import to a
temporary snapshot, then cloned the snapshot and setup the properties. This
failed when attempting to import replication streams with questionable
error.

libzfs's zfs_receive is a much better fit here, so we now use it instead
with the destination dataset and let libzfs take care of the dirty details.
be_import is greatly simplified as a result.

Reported by: Marie Helene Kvello-Aune <freebsd@mhka.no>
MFC after: 1 week


# fc13fc1c 09-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): move altroot augmentation bits around a little bit

We could perhaps have a method that does this given a dataset, but it's yet
clear that we'll always want to bypass the altroot when we grab the
mountpoint. For now, we'll refactor things a bit so we grab the altroot
length when libbe is initialized and have a common method that does the
necessary augmentation (replace with / if it's the root, return a pointer to
later in the string if not).

This will be used in some upcoming work to make be_mount work properly for
deep BEs.

MFC after: 1 week


# f08dac4e 07-Jan-2019 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Don't allow bootfs to be destroyed

Previously, the following sequence of events was feasible under some
circumstance:

bectl create test
bectl activate test
# the test BE dataset gets promoted and set as bootfs
bectl destroy test

I was unable to reproduce the destroy succeeding, but we should be rejecting
this before it even gets to libzfs because it would leave the system in an
inconsistent state. Forcing the user to be explicit as to which environment
should be activated instead is much better.

Reported by: Graham Perrin <grahamperrin@gmail.com>
MFC after: 3 days


# 4ab5187d 19-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Handle non-ZFS rootfs better

If rootfs isn't ZFS, current version will emit an error claiming so and fail
to initialize libbe. As a consumer, bectl -r (undocumented) can be specified
to operate on a BE independently of whether on a UFS or ZFS root.

Unbreak this for the UFS case by only erroring out the init if we can't
determine a ZFS dataset for rootfs and no BE root was specified. Consumers
of libbe should take care to ensure that rootfs is non-empty if they're
trying to use it, because this could certainly be the case.

Some check is needed before zfs_path_to_zhandle because it will
unconditionally emit to stderr if the path isn't a ZFS filesystem, which is
unhelpful for our purposes.

This should also unbreak the bectl(8) tests on a UFS root, as is the case in
Jenkins' -test runs.

MFC after: 3 days


# af43c24d 18-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Properly account for altroot when creating new BEs

Previously we would blindly copy the 'mountpoint' property, which includes
the altroot. The altroot needs to be snipped off prior to setting it on the
new BE, though, or you'll end up with a new BE and a mountpoint of /mnt with
altroot=/mnt

MFC after: 3 days


# cc624025 18-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

bectl(3)/libbe(3): Allow BE root to be specified

Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:

- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)

Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.

bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt

this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.

MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029


# 51aecc89 17-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): rewrite init to support chroot usage

libbe(3) currently uses zfs_be_root and locates which of its children is
currently mounted at "/". This is reasonable, but not correct in the case of
a chroot, for two reasons:

- chroot root may be of a different zpool than zfs_be_root
- chroot root will not show up as mounted at "/"

Fix both of these by rewriting libbe_init to work from the rootfs down.
zfs_path_to_zhandle on / will resolve to the dataset mounted at the new
root, rather than the real root. From there, we can derive the BE root/pool
and grab the bootfs off of the new pool. This does no harm in the average
case, and opens up bectl to operating on different pools for scenarios where
one may be, for instance, updating a pool that generally gets re-rooted into
from a separate UFS root or zfs bootpool.

While here, I've also:
- Eliminated the check for /boot and / to be on the same partition. This
leaves one open to a setup where /boot (and consequently, kernel/modules)
are not included in the boot environment. This may very well be an
intentional setup done by someone that knows what they're doing, we should
not kill BE usage because of it.

- Eliminated the validation bits of BEs and snapshots that enforced
'mountpoint' to be "/" -- this broke when trying to operate on an imported
pool with an altroot, but we need not be this picky.

Reported by: philip
Reviewed by: philip, allanjude (previous version)
Tested by: philip
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18012


# 8d4ce358 10-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Set canmount properly when activating a new BE

The previously activated BE should have canmount=noauto set on it upon
activation of the new BE, but we previously did not touch canmount on either
old or new BE.

PR: 233113
MFC after: 3 days


# 4635676d 01-Nov-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Don't promote non-cloned BEs

Most easily reproducible by attempting to activate the currently activated
BE, one would get a "not a cloned filesystem" error instead of success or a
sane message.

PR: 232488
MFC after: 3 days


# 0cadc427 01-Oct-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix BE activation promoting activated BE

This allows older BEs to be destroyed as they become replaced by a BE
created from them: e.g.

bectl create -e brokenworld fixedworld
bectl activate fixedworld
bectl destroy brokenworld

Submitted by: Shawn Webb
Approved by: re (gjb)
Obtained from: HardenedBSD (5948c0581e)


# 162ec569 31-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix error handling with respect to be_exists

Some paths through be_exists will set the error state, others will not
There are multiple reasons that a call can fail, so clean it up a bit: all
paths now return an appropriate error code so the caller can attempt to
distinguish between a BE legitimately not existing and just having the wrong
mountpoint. The caller is expected to bubble the error through to the
internal error handler as needed.

This fixes some unfriendliness with bectl(8)'s activate subcommand, where
it might fail due to a bad mountpoint but the only message output is a
generic "failed to activate" message.

Approved by: re (gjb)


# 8369ba42 24-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Make consistent with beadm

vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.

Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.

A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.

Approved by: re (kib)


# 5b7803a9 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Impose dataset length restrictions on boot env name validation

Previously, we only validated names for character restrictions. This is
helpful, but we should've also checked length restrictions- dataset names
must be restricted to MAXNAMELEN.

While here, move validation before doing a bunch of concatenations and fix
error handling in be_rename. It was previously setting the error state based
on return value from a libzfs function, which is wrong: libzfs errors don't
necessarily match cleanly to libbe errors. This would cause the assertion in
be_error to hit when the error was printed.


# a8e44f4d 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Prefer safer versions of strcat/strcpy

Or, in the activate case, just use snprintf since that's effectively what
we're doing anyways.


# 55b0e92b 16-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Hit rewind on a bunch of off-by-ones

While here, use sizeof() in some places that it makes sense to reduce room
for error and prefer strlcpy to strncpy


# cc4deabc 14-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix leaky faucets

Amongst them:
- Resource leaks
- Logically dead code
- Unused values
- Null termination issues

Reported by: asomers (pointer to Coverity), Coverity
CID: 1394777, 1394791, 1394830, 1394844, 1394872, 1394894,
CID: 1394900, 1394907, 1394950, 1394965


# 00ece7c2 12-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Light typo fix/word addition


# 1b057aac 12-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Fix be_import to delete temp snapshot

Deleting the temp snapshot isn't immediately possible because it's the
origin of the newly imported boot environment. However, this is trivially
solved by opening the new boot environment and promoting it. The roles are
now reversed and the temp snapshot/dataset may be completely destroyed.

Remove the BUGS from libbe(3) and bectl(8).


# d2a6bc9f 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Document the import bug...


# 3d1a1f2c 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Kill off the 'add' functionality for now

The mostly-undocumented 'add' functionality, from initial read-through, is
intended for construction of deep ("bdrewery style") boot environments.
However, it's mostly broken at this point. `#if SOON` it out on both sides
so that we're not exposing a broken API/feature.

Work will resume on it in due time.


# c65a2111 10-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): More error handling bits

be_add_child functionality gets split out into separate places as a bonus.
A lot of places here we'll gloss over libzfs errors, because they shouldn't
be happening given the conditions that we're operating under. "Unknown
error" is what I'm intending to use for the moment to indicate an
exceptional circumstance- exceptional enough that we can't tell the consumer
did because we're not so certain that they did anything.


# 6d4b1d24 09-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Plug some holes, do some more proper error returns

For those returning just -1 before, have them set ERR_UNKNOWN for now.


# 73c3d608 09-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): more small cleanup, const'ify and light style(9)


# 506f5fdf 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Some more light error handling...


# 2989df09 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Clarify some errors

While here, fix a bug with 'rename' that checked the wrong name for being
the active BE.


# b6e7c421 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Standardize $FreeBSD$ IDs


# b179da01 07-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/bectl(8): Standardize copyright headers

- File names don't necessarily need to be repeated
- Add SPDX tags
- Add a missing copyright for Kyle Kneitinger in bectl.8, originally written
by him in GSoC 2017; his standard copyright notice has been copied from
other files within the same directory to remain consistent with how he
clearly wished to portray it


# 920abf4d 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Destroy all children of a BE dataset, too

This fixes destruction of a deep BE returning an EBUSY because child
datasets still exist.


# 3f4e6c96 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

This snippet is no longer from zfsbootcfg


# d06f7103 06-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Rewrite activate temp bits to rely less on loader

Loader is still relied upon at the beginning of libbe to specify the be
root, but we can derive from that the primary zpool and any vdevs that we
need to set nextboot bits on.

This lets me successfully `bectl activate -t test`, but UEFI loader doesn't
quite yet understand so it's effectively defunct.


# 9b1662e6 04-Aug-2018 Kyle Evans <kevans@FreeBSD.org>

bectl: Implement -D ("space if origin datasets were deleted")

This also accomplishes the following:

- Proxy through zfs_nicenum as be_nicenum, because it looks better than
humanize_number and would presumably be useful to other libbe consumers.

- Rename be_get_snapshot_props to be_get_dataset_props, make it more useful


# b29bf2f8 25-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3)/be(8): Drop WARNS overrides, fix all fallout

Based on the idea that we shouldn't have all-new library and utility going
into base that need WARNS=1...

- Decent amount of constification
- Lots of parentheses
- Minor other nits


# ff8676cc 25-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Add nextboot flag to returned BE information


# bfe0869c 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): make style consistent with what I'll use going forward


# ee16b7c9 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Find rootfs instead by enumerating child datasets of BE root

This makes us more resilient to a rename of the bootfs, but still wouldn't
withstand pool renames or guid renames. More importantly, this allows `bectl
create <foo>` work out of the box to create a boot environment based on the
currently booted one.


# c3a34c08 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

libbe(3): Disambiguate 'active' a little bit, add 'bootfs'

- Rename 'active' to 'rootfs', which is used in other places to describe the
currently booted (or about to be booted) BE.

- Add 'bootfs', which indicates the next boot environment to be booted. This
is pulled from the BOOTFS zpool property.

- Go ahead and keep an open handle to the active zpool. We might need to
enumerate datasets, get properties, and set properties (e.g. bootfs)
throughout other libbe bits, and a single handle isn't overly expensive.


# 28f16a0f 24-Jul-2018 Kyle Evans <kevans@FreeBSD.org>

Import libbe(3)/be(1) from socsvn/soc2017/kneitinger/libbe-head