libnvmf: Add internal library to support NVMe over Fabrics

libnvmf provides APIs for transmitting and receiving Command and
Response capsules along with data associated with NVMe commands.
Capsules are represented by 'struct nvmf_capsule' objects.

Capsules are transmitted and received on queue pairs represented by
'struct nvmf_qpair' objects.

Queue pairs belong to an association represented by a 'struct
nvmf_association' object.

libnvmf provides additional helper APIs to assist with constructing
command capsules for a host, response capsules for a controller,
connecting queue pairs to a remote controller and optionally
offloading connected queues to an in-kernel host, accepting queue pair
connections from remote hosts and optionally offloading connected
queues to an in-kernel controller, constructing controller data
structures for local controllers, etc.

libnvmf also includes an internal transport abstraction as well as an
implementation of a userspace TCP transport.

libnvmf is primarily intended for ease of use and low-traffic use cases
such as establishing connections that are handed off to the kernel.
As such, it uses a simple API built on blocking I/O.

For a host, a consumer first populates an 'struct
nvmf_association_params' with a set of parameters shared by all queue
pairs for a single association such as whether or not to use SQ flow
control and header and data digests and creates a 'struct
nvmf_association' object. The consumer is responsible for
establishing a TCP socket for each queue pair. This socket is
included in the 'struct nvmf_qpair_params' passed to 'nvmf_connect' to
complete transport-specific negotiation, send a Fabrics Connect
command, and wait for the Connect reply. Upon success, a new 'struct
nvmf_qpair' object is returned. This queue pair can then be used to
send and receive capsules. A command capsule is allocated, populated
with an SQE and optional data buffer, and transmitted via
nvmf_host_transmit_command. The consumer can then wait for a reply
via nvmf_host_wait_for_response. The library also provides some
wrapper functions such as nvmf_read_property and nvmf_write_property
which send a command and wait for a response synchronously.

For a controller, a consumer uses a single association for a set of
incoming connections. A consumer can choose to use multiple
associations (e.g. a separate association for connections to a
discovery controller listening on a different port than I/O
controllers). The consumer is responsible for accepting TCP sockets
directly, but once a socket has been accepted it is passed to
nvmf_accept to perform transport-specific negotiation and wait for the
Connect command. Similar to nvmf_connect, nvmf_accept returns a newly
construct nvmf_qpair. However, in contrast to nvmf_connect,
nvmf_accept does not complete the Fabrics negotiation. The consumer
must explicitly send a response capsule before waiting for additional
command capsules to arrive. In particular, in the kernel offload
case, the Connect command and data are provided to the kernel
controller and the Connect response capsule is sent by the kernel once
it is ready to handle the new queue pair.

For userspace controller command handling, the consumer uses
nvmf_controller_receive_capsule to wait for a command capsule.
nvmf_receive_controller_data is used to retrieve any data from a
command (e.g. the data for a WRITE command). It can be called
multiple times to split the data transfer into smaller sizes.
nvmf_send_controller_data is used to send data to a remote host in
response to a command. It also sends a response capsule indicating
success, or an error if an internal error occurs. nvmf_send_response
is used to send a response without associated data. There are also
several convenience wrappers such as nvmf_send_success and
nvmf_send_generic_error.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44710

src.libnames.mk: fix LIBPFCTL definition

Following the convention used in the rest of this file, ${LIBPFCTLDIR}
should refer to the directory, and ${LIBPFCTL} to the library itself.

Instead, both values were assigned to ${LIBPFCTL}, and ${LIBPFCTLDIR} was
not set at all.

This appears to be a simple typo and not a deliberate choice, so fix it
by assigning the directory name to ${LIBPFCTLDIR} instead.

diff: Integrate libdiff from OpenBSD GoT.

This adds support for two new diff algorithms, Myers diff and Patience
diff.

These algorithms perform a different form of search compared to the
classic Stone algorithm and support escapes when worst case scenarios
are encountered.

Add the -A flag to allow selection of the algorithm, but default to
using the new Myers diff implementation.

The libdiff implementation currently only supports a subset of input and
output options supported by diff. When these options are used, but the
algorithm is not selected, automatically fallback to the classic Stone
algorithm until support for these modes can be added.

Based on work originally done by thj@ with contributions from kevans@.

Sponsored by: Klara, Inc.
Reviewed by: thj
Differential Revision: https://reviews.freebsd.org/D44302

libfdt: Make an internal FDT library available

This will be used by bhyve to build a device tree when booting arm64
guests.

Reviewed by: corvink, jhb
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40994

lib{c,sys}: move auxargs more firmly into libsys

Continue to filter the public interface (elf_aux_info()), but entierly
relocate the private interfaces (_elf_aux_info(),
__init_elf_aux_vector(), and __elf_aux_vector) to libsys.

This ensures that rtld updates the correct (only) copy of
__elf_aux_vector. After 968a18975adc9c2a619bb52aa2f009de99fc9e24
updates were confused and __getosreldate was failing, causing
the system to fall back to compat compat12 syscalls in some cases.

Return to explicitly linking libc to libsys and link libthr with libc
and libsys (in that order).

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D43910

libc: don't directly link libsys

It is sufficent to add it as a filter.

Reported by: kib
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D43781

libc: link libsys as a auxiliary filter library

At runtime, when rtld loads libc it will also load libsys. For each
symbol that is present in both, the libsys one will override the libc
one. It continues to be the case that program need only link against
libc (usually implicitly). The linkage to libsys is automatic.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908

libsys: plumb in to build

libsys provides the FreeBSD kernel interface (auxargs, system calls,
vdso). It can be linked directly for programs using a non-standard
libc and will later be linked as a filter library to libc providing
the actual system call implementation.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908

Revert "kerberos: Fix numerous segfaults when using weak crypto"

This revision breaks Linux and MacOS cross builds because
TARGET_ENDIANNESS is not define during bootstrapping on these
platforms.

I think the correct approach would be to separate the new
fbsd_ossl_provider_load() and unload functions into their own
library (instead of libroken). This avoids the less desirable
option of including bsd.cpu.mk in secure/lib/Makefile.common,
which does build but could complicate future work.

Reported by: jrtc27

This reverts commit cb350ba7bf7ca7c4cb97ed2c20ab45af60382cfb.

kerberos: Fix numerous segfaults when using weak crypto

Weak crypto is provided by the openssl legacy provider which is
not load by default. Load the legacy providers as needed.

When the legacy provider is loaded into the default context the default
provider will no longer be automatically loaded. Without the default
provider the various kerberos applicaions and functions will abort().

PR: 272835
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D43009
Tested by: netchild, Joerg Pulz <Joerg.Pulz@frm2.tum.de>

dtrace: Add the 'oformat' libdtrace option

This option can be used to specify a format to use in DTrace output.
The following formats are supported:
- json
- xml
- html
- none (default DTrace output)

This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:

- dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
- dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
- dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
- dtrace_set_outfp(FILE *) -- sets the output file for oformat.
- Ensure that oformat is correctly checked in the drop handler and record
processing callbacks.

This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.

Reviewed by: markj
Discussed with: phil
MFC after: 2 months
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41745

libbsddialog: delete formw dependency

libbsddialog >= 0.3 has a built-in form implementation so delete
formw dependency.

Approved by: bapt (share/mk maintainer)
Differential Revision: https://reviews.freebsd.org/D42167

zfs: merge openzfs/zfs@804414aad

Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress

Obtained from: OpenZFS
OpenZFS commit: 804414aad224b432590afe3f9ec114ffb49e0f13

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

libtacplus: Allow additional AV pairs to be configured.

* Replace hand-rolled input tokenizer with openpam_readlinev() which supports line continuations and has better quoting and escaping.
* Simplify string handling by merging struct clnt_str and struct srvr_str into just struct tac_str.
* Each server entry in the configuration file can now have up to 255 AV pairs which will be appended to the ones returned by the server in response to a successful authorization request.

This allows nss_tacplus(8) to be used with servers which do not provide identity information beyond confirming the existence of the user.

This adds a dependency on libpam, however libtacplus is currently only used by pam_tacplus(8) (which is already always used with libpam) and the very recently added nss_tacplus(8) (which is extremely niche). In the longer term it might be a good idea to split this out into a separate library.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: pauamma_gundo.com, markj
Differential Revision: https://reviews.freebsd.org/D40285
Relnotes: yes

bhyve: Move libcasper dependecy to lib9p

libcasper(3) is not used in bhyve. So move dependency to the appropriate
place.

Reviewed by: markj
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38905

Retire WITHOUT_CXX option

Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.

This reverts commit adc3c128c6603054586a993d117e5dd808deac17.

Reviewed by: brooks, kevans, jhb (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33108

Allow site.*.mk to augment local.*.mk and src.*.mk

Add some extra customization points so that FreeBSD build
can be adapted to local requirements.
We use these to minimize changes to share/mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37617

Put OPIE to rest.

Differential Revision: https://reviews.freebsd.org/D36592

libc: add test case for qsort_b(3)

Reviewed by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36463

src.libnames.mk: NEED/WANT_COMPAT+static support

Look in _LIB_OBJTOP for all static libraries not just INTERNALLIBs. In
normal operation this is a no-op, but improves the consistency of this
file.

Reviewed by: imp, jhb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35822

libproc: replace home grown crc32 with zlib implementation.

MFC after: 2 weeks
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D35766

src.libnames.mk: add more lib variables

Submitted by: hselasky
Reviewed by: jhb
Sponsored by: NVIDIA Networking
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D32551

irdma: Add RDMA driver for Intel(R) Ethernet Controller E810

This is an initial commit for RDMA FreeBSD driver for Intel(R) Ethernet
Controller E810, called irdma. Supporting both RoCEv2 and iWARP
protocols in per-PF manner, RoCEv2 being the default.

Testing has been done using krping tool, perftest, ucmatose, rping,
ud_pingpong, rc_pingpong and others.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: #manpages (pauamma_gundo.com) [documentation]
MFC after: 1 week
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D34690

src.libnames: Replace redundant += with =

The += for unique assignments is equivalent to =. Make these confusing
assignments simply assignments.

Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35242

src.libnames.mk: add a comment explaining libssp_nonshared

libssp_nonshared is a special case for (only) i386 and power*. Add a
comment explaining why, based on the original commit message that added
it.

MFC after: 1 week
Fixes: 0f61170882cb ("libssp_nonshared: use only on i386 and ppc")
Sponsored by: The FreeBSD Foundation

Allow downstream projects to easily add private and internal libs

Allow projects based on the FreeBSD tree to append to _PRIVATELIBS
and _INTERNALLIBS by simply maintaining their own lists of
LOCAL_PRIVATELIBS and LOCAL_INTERNALLIBS, respectively.

Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D33901

Add an internal libiscsiutil library.

Move some of the code duplicated between ctld(8) and iscsid(8) into a
libiscsiutil library.

Sharing the low-level PDU code did require having a
'struct connection' base class with a method table to permit separate
initiator vs target behavior (e.g. in handling proxy PDUs).

Reviewed by: mav, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33544

src.libnames.mk: Extend _DP_archive hack when bootstrapping on non-FreeBSD

On Linux, libcrypto is available so a non-WITHOUT_OPENSSL build happens
to work, but on macOS it isn't available to be linked against (though
there is a versioned dylib for /usr/bin/openssl etc), and neither have
libmd so would both be broken under WITHOUT_OPENSSL. Since we're using
the system libarchive, further extend the hack to ensure _DP_archive is
empty.

Fixes: ed4050750c1a ("src.libnames.mk: Add hack to workaround libarchive not being bootstrapped")
MFC after: 1 week

src.libnames.mk: Include dependencies when bootstrapping from non-FreeBSD

When bootstrapping on FreeBSD we use -DNO_SHARED so this case is already
handled correctly, but on non-FreeBSD we set NO_SHARED to no in
Makefile.boot.pre as not all OSes have static libraries available. As a
result, users of libdwarf fail to link during the cross tools stage due
to the newly-introduced dependency of libdwarf on libz.

This should perhaps be reworked to instead leave NO_SHARED as yes but
have an override (either implicit in bsd.prog.mk, or explicit via a new
variable) to turn off just the use of -static.

MFC after: 1 week

src.libnames.mk: Add hack to workaround libarchive not being bootstrapped

We don't currently honour _DP_foo when bootstrapping on non-FreeBSD, and
so none of these values matter, but the next commit will change that as
we do need to pull in dependencies for libdwarf. We should really be
bootstrapping our libarchive for ar anyway rather than using the host's,
as well as have a better way to communicate to src.libnames.mk whether
or not a library is being bootstrapped.

MFC after: 1 week

libdwarf: Support consumption of compressed ELF sections

Automatically decompress zlib-compressed debug sections when loading
them. This lets ctfcovert work on userland code after commit
c910570e7573 ("Use compressed debug in standalone userland debug files
by default").

Reported by: avg
Reviewed by: avg, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33139

bsddialog: import new TUI widget and utility

bsddialog is an attempt to write in permissive license a replacement for
libdialog.

While it is still in early stage it is good enough to already be used in
many areas, it is imported as private lib until it matures enough to be
considered as having a stable ABI

ncurses: register formw as usable lib for LIBADD

cddl: fix missing ZFS library dependencies

In 9fae47666 zfsd got a libspl dependency to avoid undefined references.
However that workaround did not help external consumers of libzfs_core.

Fix all missing dependencies lld 13 and the rtld complain about.

Reviewed by: freqlabs, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32521

Add libfido2 to the build

From https://github.com/Yubico/libfido2:

libfido2 provides library functionality and command-line tools to
communicate with a FIDO device over USB, and to verify attestation
and assertion signatures.

libfido2 supports the FIDO U2F (CTAP 1) and FIDO 2.0 (CTAP 2)
protocols.

libfido2 will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32448

Add libcbor to the build

From https://github.com/PJK/libcbor:

libcbor is a C library for parsing and generating CBOR, the general-
purpose schema-less binary data format.

libcbor will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

cbor_export.h and configuration.h were generated by the upstream CMake
build. We could create them with bmake rules instead (as NetBSD has
done) but this is a fine start.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h, and there is
no need for libcbor without libfido2.

Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32347

ncurses: chase dependency changes in the source tree

Differential Revision: https://reviews.freebsd.org/D32098

ncurses: split libtinfo from libncurses

many external program expects libncurses to not be provided as a single
library. Instead of fixing all ports, distribute ncurses the way
upstream distributes it

Turn libncursesw.so into a ldscript which will link automatically as
needed to libtinfow so so this change is seamless at compile time.

Differential Revision: https://reviews.freebsd.org/D32098

Simplify and speed up the kyua build

Instead of having multiple kyua libraries, just include the files as part
of usr.bin/kyua. Previously, we would build each kyua source up to four
times: once as a .o file and once as a .pieo. Additionally, the kyua
libraries might be built again for compat32. As all the kyua libraries
amount to 102 C++ sources the build time is significant (especially when
using an assertions enabled compiler). This change ensures that we build
306 fewer .cpp source files as part of buildworld.

Reviewed By: brooks
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30967

libcasper: Create a minimal cap_netdb service

Create a casper service for netdb functions.
Initially only cap_getprotobyname is implemented.

This is needed for capsicumizing sockstat.

Reviewed by: oshogbo, bcr (manpages)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D24832

wpa: Restructure wpa build

The current WPA build assumes a flat namespace. However the latest sources
from w1.fi now have a duplicate config.c, in two separate subdirectories.
The flat namespace will overwrite config.o with the output from the most
recently modified config.c, of which there are two of them.

This commit resolves this problem by building each component in
wpa's src subdirectory tree into its own .a archive, just as the w1.fi
upstream build as used by the port does. The advantages of this approach
are:

1. Duplicate source file names, i.e. config.c in the wpa_supplicant
direcory and another config.c in src/utils in the next wpa
will result in both compiles writing to the same .o file.

2. This restructure simplifies maintanence. A develper needs only to add
new files as identified by git status in the vendor branch to the
appropriate Makefile within the usr.sbin/wpa tree. This also reduces
time required to prepare a new import and should reduce error.

3. The new wpa build structure more closely represents the build as
performed by the upstream tarball.

This is in preparation for the next wpa update from w1.fi.

Reviewed by: philip
Tested by: philip
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D30372

src.libnames.mk: Remove redundant am-utils library definition

r358821 (13f7dbe822d5f447c6137dc0d4ff838fb7050dbe) retired amd(8).
This commit cleans up an artifact left in src.libnames.mk.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D29914

pfctl: Move ioctl abstraction functions into libpfctl

Introduce a library to wrap the pf ioctl interface.

MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29562

Remove the MK_LIBCPLUSPLUS option

This option has been equivalent to any form of C++ support since libstdc++
was removed. Therefore, replace all MK_LIBCPLUSPLUS uses with MK_CXX.

Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D27974

ncurses: only keep the version with widechar support

Only keep the widechar version of ncurses as libncursesw.so.9

Keep the old name to avoid breaking the ABI compatibility (the non
widechar version libncurses.so.9 is not binary compatible with
libncursesw.so.9) since all ports and base are already only linking
against the widechar version we can simply remove libncurses.so.9

Since the .9 version only lived in the dev branch and never ended in a
release, it is simply removed and not added to any binary compat
package.

Add symlinks to keep build time compatibility for anyone linking against
-lncurses

build: remove the option to build gnugrep

Unconditionally install bsdgrep as grep, bootstrap or not. Remove all
build glue and stop installing both gnugrep and libgnuregex now that
all consumers of the latter are gone.

Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D27732

Add virtio-9p (aka VirtFS) filesystem sharing to bhyve.

VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete support
for 9P2000.L protocol, except for the extended attribute support. It has
been verified to work with the qemu-kvm hypervisor.

Reviewed by: rgrimes, emaste, jhb, trasz
Approved by: trasz (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: Conclusive Engineering (development), vStack.com (funding)
Differential Revision: https://reviews.freebsd.org/D10335

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

libifconfig now depends on libm due to usage of log10().

ld.bfd in particular requires -lm to come after libifconfig on the
command line when linking rescue.

Reviewed by: freqlabs, adrian
Differential Revision: https://reviews.freebsd.org/D26258

lib: add libnetmap

This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCTRL) or the simple header-only-library
netmap_user.h (e.g. nm_open(), nm_close(), nm_mmap() etc.)

The new library offers more functionalities than netmap_user.h:
- Support for complex netmap options, such as external memory
allocators or per-buffer offsets. This opens the way to future
extensions.
- More flexibility in the netmap port bind options, such as
non-numeric names for pipes, or the ability to specify the netmap
allocator that must be used for a given port.
- Automatic tracking of the netmap memory regions in use across the
open ports.

At the moment there is no man page, but the libnetmap.h header file
has in-depth documentation.

Reviewed by: hrs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26171

libzfs: Add missing crypto dependency

libzfs_crypto.c uses PKCS5_PBKDF2_HMAC_SHA1 from libcrypto.

Reported by: John Kennedy
Sponsored by: iXsystems, Inc.

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

libcasper: Introduce cap_net a network service for Casper.

Reviewed by: emaste, markj (previous version), bcr (man page)
Differential Revision: https://reviews.freebsd.org/D24688

gtest: link against libregex for GNU extensions

gtest tests want to use \w ([[:alnum:]]) at the very least, which was
causing them to fail after r363679.

Start linking against libregex so that this shorthand is implemented.

PR: 248452

liblzma: Make liblzma use libmd implementation of SHA256.

MFC after: 2 weeks
PR: 200142

rename in-tree libevent v1 to libevent1

r316063 installed pf's embedded libevent as a private lib, with headers
in /usr/include/private/event. Unfortunately we also have a copy of
libevent v2 included in ntp, which needed to be updated for compatibility
with OpenSSL 1.1.

As unadorned 'libevent' generally refers to libevent v2, be explicit that
this one is libevent v1.

Reviewed by: vangyzen (earlier)
Differential Revision: https://reviews.freebsd.org/D17275

libauditd: make it a PRIVATELIB

According to the upstream man page (which we don't install), none of
libauditd's symbols are intended to be public. Also, I can't find any
evidence for a port that uses libauditd. Therefore, we should treat it like
other such libraries and use PRIVATELIB.

Reported by: phk
Reviewed by: cem, emaste
MFC after: 2 weeks

Do not claim libbearssl et al are INTERNALLIB

If INTERNALLIB is defined we need PIE and bsd.incs.mk is
not included.

PR: 245189
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org//D24233

Improve LIBADD/_DP_* for kyua libraries.

This fixes build with ld.bfd as the linker (e.g. on powerpc).

This corrects a bug in D24103.

Sponsored by: DARPA

Import the kyua test framework.

Having kyua in the base system will simplify automated testing in CI and
eliminates bootstrapping issues on new platforms.

The build of kyua is controlled by WITH(OUT)_TESTS_SUPPORT.

Reviewed by: emaste
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24103

Add liblutok a lightweight C++ API for lua.

It is added an INTERNALLIB and not installed. It will be used by kyua.

This is a preparatory commit for D24103.

Reviewed by: emaste
Obtained from: CheriBSD
Sponsored by: DARPA

Add an internal liblua and use it in flua.

The new liblua will be used in a forthcoming import of kyua.

Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24090

remove GCC 4.2.1 build infrastructure

As described in Warner's email message[1] to the FreeBSD-arch mailing
list we have reached GCC 4.2.1's retirement date. At this time all
supported architectures either use in-tree Clang, or rely on external
toolchain (i.e., a contemporary GCC version from ports).

GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later
that year, in r171825. GCC has served us well, but version 4.2.1 is
obsolete and not used by default on any architecture in FreeBSD. It
does not support modern C and does not support arm64 or RISC-V.

Thanks to everyone responsible for maintaining, updating, and testing
GCC in the FreeBSD base system over the years.

So long, and thanks for all the fish.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR: 228919
Reviewed by: brooks, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23124

libssp_nonshared: use only on i386 and ppc

libssp_nonshared.a defines one symbol, __stack_chk_fail_local. This
is used only on i386 and powerpc; other archs emit calls directly to
__stack_chk_fail. Simplify linking on other archs by omitting it.

PR: 242941 [exp-run]

Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943

Allow OBJDIR to be overridden for LIB*DIR variables.

This will allow us to link against internal libraries when building
programs for the system's LIBCOMPAT ABI.

Reviewed by: bdrewery
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL

Introduce stats(3), a flexible statistics gathering API.

This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof). Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs. The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by: sef (earlier version)
Obtained from: Netflix
Relnotes: yes
Sponsored by: Klara Inc, Netflix
Differential Revision: https://reviews.freebsd.org/D20477

Add the missing bits for LIBADD to properly function now that
libarchive is linked to libzstd

Pointy hat: bapt
Reported by: antoine

Remove cap_random(3).

Now that we have a way to obtain entropy in capability mode
(getrandom(2)), libcap_random is obsolete. Remove it.

Bump __FreeBSD_version in case anything happens to use it, though I've
found no consumers.

Reviewed by: delphij, emaste, oshogbo
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21033

Remove NAND and NANDFS support

NANDFS has been broken for years. Remove it. The NAND drivers that
remain are for ancient parts that are no longer relevant. They are
polled, have terrible performance and just for ancient arm
hardware. NAND parts have evolved significantly from this early work
and little to none of it would be relevant should someone need to
update to support raw nand. This code has been off by default for
years and has violated the vnode protocol leading to panics since it
was committed.

Numerous posts to arch@ and other locations have found no actual users
for this software.

Relnotes: Yes
No Objection From: arch@
Differential Revision: https://reviews.freebsd.org/D20745

MFhead@r344786

Enable build of libbearssl

Reviewed by: emaste
Sponsored by: Juniper Networks
Differential Revision: D16337

Make libifconfig INTERNALLIB

Instead of PRIVATELIB + NO_PIC. This avoids the need for the wlandebug
PIE special case added in r344211, and provides a stronger guarantee
against 3rd party software coming to depend on the API or ABI.

If / when we declare the API/ABI to be stable we can make it a normal
library.

Discussed with: bapt
Sponsored by: The FreeBSD Foundation

Fix typos

LIBGMOCK_MAIN and LIBGTEST_MAIN were missing corresponding underscores in the
variable names.

MFhead@r344270

Use make's :tl instead of checking "no" and "NO"

Suggested by: kevans
Reviewed by: kevans

Add WITH_PIE knob to build Position Independent Executables

Building binaries as PIE allows the executable itself to be loaded at a
random address when ASLR is enabled (not just its shared libraries).

With this change PIE objects have a .pieo extension and INTERNALLIB
libraries libXXX_pie.a.

MK_PIE is disabled for some kerberos5 tools, Clang, and Subversion, as
they explicitly reference .a libraries in their Makefiles. These can
be addressed on an individual basis later. MK_PIE is also disabled for
rtld-elf because it is already position-independent using bespoke
Makefile rules.

Currently only dynamically linked binaries will be built as PIE.

Discussed with: dim
Reviewed by: kib
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18423

Merge build glue for libraries and tests done on github

I need to doublecheck my work vs the port, but I believe that this covers the
initial integration of all upstream tests.

Ref: https://github.com/ngie-eign/freebsd/tree/googletest-integration

libcasper: introduce cap_fileargs service

cap_fileargs is a Casper service which helps to sandbox applications that need
access to the filesystem namespace. The main purpose of the service is to make
easy to capsicumize applications that works on multiple files passed in argv.

We have a couple example of using it but we still treat this service as an
experimental one.

Reviewed by: emsate (previous version), jonathan (partially)
Discussed with: many
Differential Revision: https://reviews.freebsd.org/D14407

Add libssl to libldns for DANE.

Approved by: re (gjb)

Link libcrypto with pthread.

libbe(3): Move build goop back out of cddl/

Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.

Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.

IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.

libbe(3): Add to cddl build, adjust src.libnames.mk as needed

src.libnames.mk: Make libbe entry consistent with others

Perhaps using libbe.a from "${DESTDIR}${LIBDIR}" might have been the
prevailing technique at one point, but nowadays it appears to be to
preferred to spell this as "${OBJTOP}/lib" -- make it so.

Import libbe(3)/be(1) from socsvn/soc2017/kneitinger/libbe-head

Rework ofed build.

Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from: bdrewery
Reviewed by: bdrewery, hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D15648

MFV r333789: libpcap 1.9.0 (pre-release)

MFC after: 1 month
Sponsored by: Mellanox Technologies

Created static libc PIC/no-SSP library to be used by rtld.

Rtld is not compatible with SSP, and since we link libc_pic.a to rtld
to have the basic support like memory and string copy functions, we
have to both carefully limit libc use, and to provide the ssp support
shims. This change makes the libc use in rtld more straighforward but
still limited, and allows to remove the shims, to be done in the next
commit.

Submitted by: Luis Pires
Reviewed by: bdrewery, brooks
Differential revision: https://reviews.freebsd.org/D15283

Add new shared library -- libopencsd.

OpenCSD is an ARM CoreSight(tm) trace packets decoder.

- Connect libopencsd to the arm64 build.
- Install opencsd headers to /usr/include/opencsd/

Sponsored by: DARPA, AFRL

Add new shared library -- libipt.

libipt is the Intel Processor Trace (Intel PT) packets decoder.

- Include libipt to amd64 build.
- Install libipt headers to /usr/include/libipt/

Sponsored by: DARPA, AFRL

Remove libreadline from the source tree, all consumers but gdb
has been switched to libedit long ago, libreadline was built as an
internallib for a while and kept only for gdbtui which was broken using
libreadline.

Since gdb has been mostly deorbitted in all arches, gdbtui was only installed
on arm and sparc64, given it has been removed, gdb has been switched to use
libedit, no consumers are left for libreadline. Thus this removal

Add libregex, connect it to the build

libregex is a regex(3) implementation intended to feature GNU extensions and
any other non-POSIX compliant extensions that are deemed worthy.

These extensions are separated out into a separate library for the sake of
not cluttering up libc further with them as well as not deteriorating the
speed (or lack thereof) of the libc implementation.

libregex is implemented as a build of the libc implementation with LIBREGEX
defined to distinguish this from a libc build. The reasons for
implementation like this are two-fold:

1.) Maintenance- This reduces the overhead induced by adding yet another
regex implementation to base.

2.) Ease of use- Flipping on GNU extensions will be as simple as linking
against libregex, and POSIX-compliant compilations can be guaranteed with a
REG_POSIX cflag that should be ignored by libc/regex and disables extensions
in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when
implemented in this fashion.

Tests are added for future functionality, but left disconnected for the time
being while other testing is done.

Reviewed by: cem (previous version)
Differential Revision: https://reviews.freebsd.org/D12934

Look for libraries in OBJDIR when building outside of buildworld.

This allows build testing more easily without establishing a sysroot
or installing the files.

Sponsored by: Dell EMC

Merge ^/head r325663 through r325841.

Fix OFED library dependencies.

Sponsored by: Mellanox Technologies

Remove no longer supported mthca driver.

Sponsored by: Mellanox Technologies

Introduce syslog service for Casper.

syslog in libc secretly reconnects to the daemon.
Another issue is that we don't have any information from openlog(3) if we
succeeded to open log or not so we don't know if we are ready
to enter cabability mode.
Because all of that we decided we need a syslog service for Caspser.

Reviewed by: bapt@
Differential Revision: https://reviews.freebsd.org/D12824

Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12718

Mark libifconfig as private library in src.libnames.mk (completes r305700)

Provide libdl.

Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.

Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.

Reviewed by: bdrewery, dim (previos version); emaste
Exp run: PR 220525, done by antoine
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D11504

Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.

Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
A LIBSTAND hack is no longer required for buildenv.

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Fix Makefiles which override LIBDIR to not add incorrect dependencies into .depend.

This fixes these cases which would rebuild every time:
make[6]: /usr/obj/usr/src/libexec/rtld-elf/tests/libpythagoras/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/libexec/rtld-elf/libm.a
make[6]: /usr/obj/usr/src/lib/libxo/tests/encoder/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libxo/libxo.a
make[7]: /usr/obj/usr/src/lib/libthr/tests/dlopen/dso/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libthr/dlopen/libpthread.a

The problem is that some Makefiles will override LIBDIR to where they want
their library to install. bsd.libnames.mk will then use ${LIBDIR} to define
where *existing* libraries are. This then leads to looking for the
libraries in the *target* place rather than the *expected* place.

We may want to expand this (and all of the other *DIR variables in bsd.own.mk)
into something like what Ports has, a PREFIX and a LOCALBASE. PREFIX being
where things are being installed to and LOCALBASE being where they already are.

For now store the default expected LIBDIR into LIBDIR_BASE and use that for
library locations.

Reported by: sbruno
MFC after: 1 week
Sponsored by: Dell EMC Isilon

Fix more incorrect library directories fix 'stale .depend' rebuilds.

Reported by: sbruno
MFC after: 3 days
Sponsored by: Dell EMC Isilon

Fix LIBAMU location to fix 'stale .depend' rebuilds in usr.sbin/amd.

This originally came in r275052.

Reported by: sbruno
MFC after: 3 days
Sponsored by: Dell EMC Isilon

Build zstandard with threading enabled

Import zstandard 1.1.4 in base

zstandard is a new compression library/tool which is very fast at
compression/decompression

For now import as a private library

Create libefivar library. This library aims to provide
the same API as the GPL'd version of this library. It implements the common
Linux API for programatically manipulating UEFI environment varibales using
the UEFI Runtime Services the kernel provides. It replaces the old efi
library since it is programmed to a different interface, but retails the
CHAR16 to UTF-8 and vice versa conversion routines. The new name is to match
Linux program's expectations.

Differential Revision: https://reviews.freebsd.org/D8128
Reviewed by: kib@, wblock@, Ganael Laplanche

Remove cruft that accidently crept in r305931

Disable GNU rcs by default

All remaining tools using rcs has been switched to directly use diff3(1):
- etcupdate(8)
- freebsd-update(8)

Note that the ident(1) tool is been already replaced long ago with a BSD
licensed version, as such it remains installed.

GNU rcs is still available from ports:
- rcs: newer GPLv3 version
- rcs57: the latest version from base (GPLv2)

Renaming libifc to libifconfig in response to feedback on initial commit of
this library. Sticking to 'libifconfig' (and 'ifconfig_' as function prefix)
should reduce chances of namespace collisions, make it more clear what the
library does, and be more in line with existing libraries.

Submitted by: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Differential Revision: https://reviews.freebsd.org/D7742
Reviewed by: cem, kp

Add libifc, a library implementing core functionality that exists in ifconfig(8) today.

libifc (pronounced lib-ifconfig) aims to be a light abstraction layer between
programs and the kernel APIs for managing the network configuration.
This should hopefully make programs easier to maintain, and reduce code
duplication.

Work will begin on making ifconfig(8) use this library in the near future.

This code is still evolving. The interface should not be considered stable until
it is announced as such.

Submitted By: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D7529

libproc: Add proc_getmodel().

This is used by libdtrace to determine the data model of target processes.
This allows for the creation of pid provider probes in 32-bit processes on
amd64.

MFC after: 1 month

librtld_db: Use the auxv to figure out where to look up loader symbols.

Previously, librtld_db just hardcoded /libexec/ld-elf.so, which isn't
correct for processes that aren't using the native ABI. With this change,
librtld_db can be used to inspect non-native processes; in particular,
dtrace -c now works for 32-bit executables on amd64.

MFC after: 1 month

Add basic blacklist build support

Reviewed by: rpaulo
Approved by: rpaulo
Relnotes: YES
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5913

Fix the library name.

Reviewed by: bdrewery

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

Fix over/under-linking in contrib/ofed.

Sponsored by: EMC / Isilon Storage Division

Add missing atf dirs.

Sponsored by: EMC / Isilon Storage Division

Move casper library entries to proper places.

Sponsored by: EMC / Isilon Storage Division

Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277

Add libpe for elfcopy(1) PE/COFF support

Sponsored by: The FreeBSD Foundation

Add comment about OBJTOP not yet being defined outside of META MODE.

It is fine for the purpose of the check though.

Sponsored by: EMC / Isilon Storage Division

Follow-up r291739: Don't suggest LDADD on private libs to use LIBADD.

This is because LDADD+=-lFOO is not the same as LDADD+=-lprivateFOO which is
what the private libs in LIBADD are.

Sponsored by: EMC / Isilon Storage Division

Start on a new library (libsysdecode) that provides routines for decoding
system call information such as system call arguments. Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.

This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it. One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events. This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events. Instead, this function only generates a string
description for known malloc() and RTLD records.

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D4537

Let termcap be used in LIBADD.

Sponsored by: EMC / Isilon Storage Division

Convert contrib/ofed libraries to LIBADD.

Sponsored by: EMC / Isilon Storage Division

Sort the LIB list.

Sponsored by: EMC / Isilon Storage Division

Add assertion for when LIBADD should be used rather than LDADD/DPADD.

Sponsored by: EMC / Isilon Storage Division

Rework unknown LIBADD assertion to be more clear and to not suggest adding
DPADD/LDADD_<foo> variables that are a special case.

Sponsored by: EMC / Isilon Storage Division

Support all of the CDDL/ZFS libraries for LIBADD.

Sponsored by: EMC / Isilon Storage Division

For INTERNALLIB always add in the corresponding _DP_ and use LIBADD in
the real build file.

This lessens the need to define DPADD_<lib> and LDADD_<lib> to just very
special cases.

Sponsored by: EMC / Isilon Storage Division

Fix a typo in a comment (spacial -> special)

Sponsored by: EMC / Isilon Storage Division

Add assertions that capture invalid configurations for new libraries.

Fix current findings, which should fix cases of NO_SHARED not building
properly.

Given libfoo:
- Ensure that a LIBFOO is set. For INTERNALLIBS advise setting this in
src.libnames.mk, otherwise bsd.libnames.mk.
- Ensure that a LIBFOODIR is properly set.
- Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
Makefile

Sponsored by: EMC / Isilon Storage Division

Don't overlink pthread to kerberos library consumers.

I'm not sure why this was here, none of these use pthread themselves and
none of the consumers are broken with removing this.

Sponsored by: EMC / Isilon Storage Division

Don't overlink pthread to libsqlite3 consumers.

At least usr.bin/mandoc was overlink.

Sponsored by: EMC / Isilon Storage Division

Fix truncation of _DP_proc and add missing libelf.

Sponsored by: EMC / Isilon Storage Division

Reduce overlinking of libdtrace consumers with libctf, libelf, libproc.

The proper place for this list is _DP_dtrace.

Due to removing the LDADD_dtrace, more LIBADD are needed in
cddl/usr.sbin/dtrace to prevent underlinking.

This fixes overlinking in cddl/usr.sbin/lockstat and
cddl/usr.sbin/plockstat.

Sponsored by: EMC / Isilon Storage Division

Don't overlink libmt consumers with libsbuf.

This change came in r281332 which was reducing overlinking in mt(1) but
currently mt(1) is linked with sbuf when it does not need it due to the
LDADD_mt+=${LDADD_sbuf}. Only libmt needs sbuf.

Add sbuf to _DP_mt so static linkage of libmt picks it up.

Sponsored by: EMC / Isilon Storage Division

libssh uses libz even without MK_LDNS

Sponsored by: EMC / Isilon Storage Division

Revert incomplete r291623.

Add assertions that capture invalid configurations for new libraries.

Fix current findings.

Given libfoo:
- Ensure that a LIBFOO is set. For INTERNALLIBS advise setting this in
src.libnames.mk, otherwise bsd.libnames.mk.
- Ensure that a LIBFOODIR is properly set.
- Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
Makefile

Sponsored by: EMC / Isilon Storage Division

Support LOCAL_LIBRARIES for LIBADD.

Sponsored by: EMC / Isilon Storage Division

Fix underlinking in lib80211 and define static dependencies in src.libnames.mk
so NO_SHARED works properly.

Reported by: Manfred Antar <null@pozo.com>
Sponsored by: EMC / Isilon Storage Division

Add missing lib declaration.

Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles(). It drops the
unused 'swapfile' argument and adds a new function pointer argument for
a symbol resolving function. Native kernels still use _fdnlist() from
libc to resolve symbols if a resolver function is not supplied, but cross
kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
Each backend is responsible for implementing kvm_read2() for a given
vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
powerpc cores).
- Add internal helper routines for the common page offset hash table used
by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
to be cross-friendly by using private constants instead of ones that
vary by platform (e.g. PAGE_SIZE). Static assertions are present when
a given backend is compiled natively to ensure the private constants
match the real ones.
- Enable all of the existing vmcore backends on all platforms. This means
that libkvm on any platform should be able to perform KVA translation
and read data from a vmcore of any platform.

Tested on: amd64, i386, sparc64 (marius)
Differential Revision: https://reviews.freebsd.org/D3341

Standardize on OBJTOP in and outside of META MODE.

Sponsored by: EMC / Isilon Storage Division

Define a LIB<NAME>DIR for every library that LIBADD provides.

This is going to be used to allow DIRDEPS to be bootstrapped off of
LIBADD/DPADD. It currently works for internal libraries which have a
DIR defined for them but also use the .a library from a src-mapped obj
directory. It can also be useful for using -L without a --sysroot per
LIBADD to use the OBJDIR version of the libraries.

I didn't review every LIBADD, so it is possible this is missing some.

Sponsored by: EMC / Isilon Storage Division

META MODE: Fix 'make bootstrap-tools'.

The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.

This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.

For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds. This has not yet been a problem due
to readelf not being built as a host tool in buildworld. This is possible
in the meta build though when building the toolchain.

Sponsored by: EMC / Isilon Storage Division

Merge from head r290483

Rename libohash to libopenbsd.

libopenbsd is an internal library which
to bring in compatibility stuff from OpenBSD.
This will allow us to bring in more
OpenBSD utilities into the FreeBSD base system.

We similarly use libnetbsd for bringing in stuff from NetBSD.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D4078

Merge from head

Make libxo depend on libutil because it uses humanize_number after r287111

Remove overlinking in lib/libxo/tests, sbin/savecore, and
usr.bin/{iscsictl,wc,xo}

PR: 203673
Sponsored by: EMC / Isilon Storage Division

Merge from head

Remove last traces of USEPRIVATELIB

Define libavl in src.libnames.mk

Register libpanel into the available libraries

Fix incorrect library path.

Use SRCTOP when defining ROOTOBJDIR

Excplicitly check rather than assume that .CURDIR is
a component of .OBJDIR
Otherwise check for OBJTOP and if that is a match use it.

Considers cases when NO_SHARED?=[no|NO] as dynamically linking

This reduces overlinking for parts of the build system where NO_SHARED is
set to no/NO

Drop libmandoc and incorporate it into the main mandoc Makefile

This simplifies maintainance of mandoc(1).
Note that the same direction was taken on OpenBSD

Register libdtrace and its direct and indirect dependencies
Register librdlt_db
Register libproc dependencies
Register libctf dependencies

Correct location for libntpevent.a.

Fix spelling of INTERNALLIBS

Style fix

Reported by: bdrewery

Rework PRIVATELIB

Now when a lib is marked as PRIVATELIB it is renamed into libprivate$foo instead
of being installed in /usr/lib/private and playing with rpath.

Also allow to install headers for PRIVATELIBS in that case the headers will be
installed in /usr/include/private/$foo

Keep the headers under a private namespace to prevent third party build system
to easily find them to ensure they are only used on purpose.

This allows for non base applications to statically link against a library in
base which is linked to a privatelib

Treating PRIVATELIBS as regular libraries allows to push them into our current
compatX packages if needed.

While here finish promotion of libevent as PRIVATELIB
Install header for bsdstat and libucl

Differential Revision: https://reviews.freebsd.org/D2365
Reviewed by: brooks, des
Discussed with: imp

Make it harder to specify invalid LIBADD by causing values without
corresponding DPADD_<lib> variables to produce a useful error message.

Differential Revision: https://reviews.freebsd.org/D2295
Reviewed by: bapt
Sponsored by: DARPA, AFRL

Fix libfifolog dependency on libz

Consumers should not need to encode fifolog's dependency on libz.
Handle it automatically in src.libnames.mk.

Differential Revision: https://reviews.freebsd.org/D2278
Reviewed by: bapt
Sponsored by: The FreeBSD Foundation

Fix overlinking in bhyve:

libvmmapi is actually needed to be linked to libutil, not bhyve nor bhyveload

Convert mt(1) and libmt to LIBADD

While here fix missing link to libbsdxml for libmt
Fix overlinking in mt(1)
Make add an indirect libmt dependency on bsdxml to allow static linking if
needed

MFV ntp 4.2.8p1 (r258945, r275970, r276091, r276092, r276093, r278284)

Thanks to roberto for providing pointers to wedge this into HEAD.

Approved by: roberto

Fix typo (_DP_grom -> _DP_geom) so applications that need to link against
libgeom do so successfully

Tested by running `geom part list` produced from a -DWITHOUT_DYNAMICROOT built
world

PR: 198078
Reported by: Eir Nym <eirnym@gmail.com>

Merge xz 5.2.0.

This brings support for multi-threaded compression. This brings close
N times faster compression where N is the number of CPU cores.
Because of this, liblzma now depends on libthr.

Soon libarchive will be modified to use the new lzma API.

Thanks to antoine@ for the exp-run.

Differential Revision: https://reviews.freebsd.org/D1786
Reviewed by: bapt

Add a new device control utility for new-bus devices called devctl. This
allows the user to request administrative changes to individual devices
such as attach or detaching drivers or disabling and re-enabling devices.
- Add a new /dev/devctl2 character device which uses ioctls for device
requests. The ioctls use a common 'struct devreq' which is somewhat
similar to 'struct ifreq'.
- The ioctls identify the device to operate on via a string. This
string can either by the device's name, or it can be a bus-specific
address. (For unattached devices, a bus address is the only way to
locate a device.) Bus drivers register an eventhandler to claim
unrecognized device names that the driver recognizes as a valid address.
Two buses currently support addresses: ACPI recognizes any device
in the ACPI namespace via its full path starting with "\" and
the PCI bus driver recognizes an address specification of
'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector
strings supported by pciconf).
- To make it easier to cut and paste, change the PnP location string
in the PCI bus driver to output a full PCI selector string rather
than 'slot=<slot> function=<func>'.
- Add a devctl(3) interface in libdevctl which provides a wrapper around
the ioctls and is the preferred interface for other userland code.
- Add a devctl(8) program which is a simple wrapper around the requests
supported by devctl(3).
- Add a device_is_suspended() function to check DF_SUSPENDED.
- Add a resource_unset_value() function that can be used to remove a
hint from the kernel environment. This is used to clear a
hint.<driver>.<unit>.disabled hint when re-enabling a boot-time
disabled device.

Reviewed by: imp (parts)
Requested by: imp (changing PCI location string)
Relnotes: yes

Add ncurses to _LIBRARIES so that libmenu is correctly linked

PR: ports/195782

Switch gpioctl(8) to LIBADD.

Build infrastructure for elftoolchain tools

Set WITH_ELFTOOLCHAIN_TOOLS in src.conf to use the elftoolchain version
of the following tools:

* addr2line
* elfcopy (strip / mcs)
* nm
* size
* strings

Reviewed by: bapt (earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1224

Update libucl to latest version

While here correctly link libucl to libm and register the dependency on libm
for static building

Remove all remnant ugly LD<lbname> but atf one until the aft framework knows
about LIBADD

Register the explicit (pthread) and implicit (for static) dependencies for
kerberos

Defines the libssh dependencies

Define missing libraries to be able to convert gnu, cddl and secure to LIBADD

Sort libraries definitions

Reported by: bjk

Remove duplicated krb5

Reported by: markj

Defined the needed library to convert usr.sbin to LIBADD

Defines all the libraries needed for usr.bin

Define libipf internal library

Fix recursive problem with libedit dependency, if you depends on ncursesw and
not on itself

Add a definition for libxo

Define compiler_rt and ssp_nonshared libraries

libvers and libsl are internal libraries

Fix make depends with libnetbsd

Add definitions of kerberos libraries

libevent is an internallib

libatf-c++ exposes libatf-c abi hence we need to explicit link to atf-c if
atf-c++ is requested

Do not overwrite any LIBNETBSD or LIBNETBSDDIR if they are already defined

Define special case for atf which libraries names are different from their
declaration

s/cft/ctf

Fix typo

Fix dependencies in case of building statically

Avoid code duplication by directly adding _INSTALLLIBS and _PRIVATELIBS into
_LIBRARIES

Reported by: emaste

Each time something links to sqlite3 it should link to pthread

Convert makefs to LIBADD

Add definition of libnetbsd in src.libnames.mk

Convert mandoc to LIBADD

While here remove the ugly LDMANDOC

Correctly specify -L for internal and private libs

Implement LIBADD

LIBADD will automatically set DPADD and LDADD when needed including their
dependencies, LIBADD automatically handles private and internal libs so that
the end user Makefile does not have to care about it.

This allows to reduce overlinking on the base system leaving the framework get
the dependencies properly.

It also allows to built components binaries statically.

To use it:

Replace:
DPADD= ${LIBARCHIVE} ${LIBSSL}
LDADD= -larchive -lssl

by:
LIBADD= archive ssl

Differential Revision: https://reviews.freebsd.org/D1209
Reviewed by: brooks imp

Define LIBSQLITE3 and LIBMANDOC

Promote libevent to lib/ level and fold ftp-proxy into its parent Makefile.
This allows us to use libevent for other application in the future.

For now libevent is still INTERNALLIB and no shared library is installed.

MFC after: 1 month

Make the libbsdstat useful again.

Make libohash a proper internallib instead of hidding it in m4

mandoc development seems to be also using ohash so prepare to share the code when
needed

Rework privatelib/internallib

Make sure everything linking to a privatelib and/or an internallib does it directly
from the OBJDIR rather than DESTDIR.
Add src.libnames.mk so bsd.libnames.mk is not polluted by libraries not existsing
in final installation
Introduce the LD* variable which is what ld(1) is expecting (via LDADD) to link to
internal/privatelib
Directly link to the .so in case of private library to avoid having to complexify
LDFLAGS.

Phabric: https://phabric.freebsd.org/D553
Reviewed by: imp, emaste

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

libtacplus: Allow additional AV pairs to be configured.

* Replace hand-rolled input tokenizer with openpam_readlinev() which supports line continuations and has better quoting and escaping.
* Simplify string handling by merging struct clnt_str and struct srvr_str into just struct tac_str.
* Each server entry in the configuration file can now have up to 255 AV pairs which will be appended to the ones returned by the server in response to a successful authorization request.

This allows nss_tacplus(8) to be used with servers which do not provide identity information beyond confirming the existence of the user.

This adds a dependency on libpam, however libtacplus is currently only used by pam_tacplus(8) (which is already always used with libpam) and the very recently added nss_tacplus(8) (which is extremely niche). In the longer term it might be a good idea to split this out into a separate library.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: pauamma_gundo.com, markj
Differential Revision: https://reviews.freebsd.org/D40285
Relnotes: yes

bhyve: Move libcasper dependecy to lib9p

libcasper(3) is not used in bhyve. So move dependency to the appropriate
place.

Reviewed by: markj
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38905

Retire WITHOUT_CXX option

Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.

This reverts commit adc3c128c6603054586a993d117e5dd808deac17.

Reviewed by: brooks, kevans, jhb (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33108

Allow site.*.mk to augment local.*.mk and src.*.mk

Add some extra customization points so that FreeBSD build
can be adapted to local requirements.
We use these to minimize changes to share/mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37617

Put OPIE to rest.

Differential Revision: https://reviews.freebsd.org/D36592

libc: add test case for qsort_b(3)

Reviewed by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36463

src.libnames.mk: NEED/WANT_COMPAT+static support

Look in _LIB_OBJTOP for all static libraries not just INTERNALLIBs. In
normal operation this is a no-op, but improves the consistency of this
file.

Reviewed by: imp, jhb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35822

libproc: replace home grown crc32 with zlib implementation.

MFC after: 2 weeks
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D35766

src.libnames.mk: add more lib variables

Submitted by: hselasky
Reviewed by: jhb
Sponsored by: NVIDIA Networking
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D32551

irdma: Add RDMA driver for Intel(R) Ethernet Controller E810

This is an initial commit for RDMA FreeBSD driver for Intel(R) Ethernet
Controller E810, called irdma. Supporting both RoCEv2 and iWARP
protocols in per-PF manner, RoCEv2 being the default.

Testing has been done using krping tool, perftest, ucmatose, rping,
ud_pingpong, rc_pingpong and others.

Signed-off-by: Eric Joyner <erj@FreeBSD.org>

Reviewed by: #manpages (pauamma_gundo.com) [documentation]
MFC after: 1 week
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D34690

src.libnames: Replace redundant += with =

The += for unique assignments is equivalent to =. Make these confusing
assignments simply assignments.

Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35242

src.libnames.mk: add a comment explaining libssp_nonshared

libssp_nonshared is a special case for (only) i386 and power*. Add a
comment explaining why, based on the original commit message that added
it.

MFC after: 1 week
Fixes: 0f61170882cb ("libssp_nonshared: use only on i386 and ppc")
Sponsored by: The FreeBSD Foundation

Allow downstream projects to easily add private and internal libs

Allow projects based on the FreeBSD tree to append to _PRIVATELIBS
and _INTERNALLIBS by simply maintaining their own lists of
LOCAL_PRIVATELIBS and LOCAL_INTERNALLIBS, respectively.

Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D33901

Add an internal libiscsiutil library.

Move some of the code duplicated between ctld(8) and iscsid(8) into a
libiscsiutil library.

Sharing the low-level PDU code did require having a
'struct connection' base class with a method table to permit separate
initiator vs target behavior (e.g. in handling proxy PDUs).

Reviewed by: mav, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33544

src.libnames.mk: Extend _DP_archive hack when bootstrapping on non-FreeBSD

On Linux, libcrypto is available so a non-WITHOUT_OPENSSL build happens
to work, but on macOS it isn't available to be linked against (though
there is a versioned dylib for /usr/bin/openssl etc), and neither have
libmd so would both be broken under WITHOUT_OPENSSL. Since we're using
the system libarchive, further extend the hack to ensure _DP_archive is
empty.

Fixes: ed4050750c1a ("src.libnames.mk: Add hack to workaround libarchive not being bootstrapped")
MFC after: 1 week

src.libnames.mk: Include dependencies when bootstrapping from non-FreeBSD

When bootstrapping on FreeBSD we use -DNO_SHARED so this case is already
handled correctly, but on non-FreeBSD we set NO_SHARED to no in
Makefile.boot.pre as not all OSes have static libraries available. As a
result, users of libdwarf fail to link during the cross tools stage due
to the newly-introduced dependency of libdwarf on libz.

This should perhaps be reworked to instead leave NO_SHARED as yes but
have an override (either implicit in bsd.prog.mk, or explicit via a new
variable) to turn off just the use of -static.

MFC after: 1 week

src.libnames.mk: Add hack to workaround libarchive not being bootstrapped

We don't currently honour _DP_foo when bootstrapping on non-FreeBSD, and
so none of these values matter, but the next commit will change that as
we do need to pull in dependencies for libdwarf. We should really be
bootstrapping our libarchive for ar anyway rather than using the host's,
as well as have a better way to communicate to src.libnames.mk whether
or not a library is being bootstrapped.

MFC after: 1 week

libdwarf: Support consumption of compressed ELF sections

Automatically decompress zlib-compressed debug sections when loading
them. This lets ctfcovert work on userland code after commit
c910570e7573 ("Use compressed debug in standalone userland debug files
by default").

Reported by: avg
Reviewed by: avg, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33139

bsddialog: import new TUI widget and utility

bsddialog is an attempt to write in permissive license a replacement for
libdialog.

While it is still in early stage it is good enough to already be used in
many areas, it is imported as private lib until it matures enough to be
considered as having a stable ABI

ncurses: register formw as usable lib for LIBADD

cddl: fix missing ZFS library dependencies

In 9fae47666 zfsd got a libspl dependency to avoid undefined references.
However that workaround did not help external consumers of libzfs_core.

Fix all missing dependencies lld 13 and the rtld complain about.

Reviewed by: freqlabs, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32521

Add libfido2 to the build

From https://github.com/Yubico/libfido2:

libfido2 provides library functionality and command-line tools to
communicate with a FIDO device over USB, and to verify attestation
and assertion signatures.

libfido2 supports the FIDO U2F (CTAP 1) and FIDO 2.0 (CTAP 2)
protocols.

libfido2 will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32448

Add libcbor to the build

From https://github.com/PJK/libcbor:

libcbor is a C library for parsing and generating CBOR, the general-
purpose schema-less binary data format.

libcbor will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

cbor_export.h and configuration.h were generated by the upstream CMake
build. We could create them with bmake rules instead (as NetBSD has
done) but this is a fine start.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h, and there is
no need for libcbor without libfido2.

Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32347

ncurses: chase dependency changes in the source tree

Differential Revision: https://reviews.freebsd.org/D32098

ncurses: split libtinfo from libncurses

many external program expects libncurses to not be provided as a single
library. Instead of fixing all ports, distribute ncurses the way
upstream distributes it

Turn libncursesw.so into a ldscript which will link automatically as
needed to libtinfow so so this change is seamless at compile time.

Differential Revision: https://reviews.freebsd.org/D32098

Simplify and speed up the kyua build

Instead of having multiple kyua libraries, just include the files as part
of usr.bin/kyua. Previously, we would build each kyua source up to four
times: once as a .o file and once as a .pieo. Additionally, the kyua
libraries might be built again for compat32. As all the kyua libraries
amount to 102 C++ sources the build time is significant (especially when
using an assertions enabled compiler). This change ensures that we build
306 fewer .cpp source files as part of buildworld.

Reviewed By: brooks
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30967

libcasper: Create a minimal cap_netdb service

Create a casper service for netdb functions.
Initially only cap_getprotobyname is implemented.

This is needed for capsicumizing sockstat.

Reviewed by: oshogbo, bcr (manpages)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D24832

wpa: Restructure wpa build

The current WPA build assumes a flat namespace. However the latest sources
from w1.fi now have a duplicate config.c, in two separate subdirectories.
The flat namespace will overwrite config.o with the output from the most
recently modified config.c, of which there are two of them.

This commit resolves this problem by building each component in
wpa's src subdirectory tree into its own .a archive, just as the w1.fi
upstream build as used by the port does. The advantages of this approach
are:

1. Duplicate source file names, i.e. config.c in the wpa_supplicant
direcory and another config.c in src/utils in the next wpa
will result in both compiles writing to the same .o file.

2. This restructure simplifies maintanence. A develper needs only to add
new files as identified by git status in the vendor branch to the
appropriate Makefile within the usr.sbin/wpa tree. This also reduces
time required to prepare a new import and should reduce error.

3. The new wpa build structure more closely represents the build as
performed by the upstream tarball.

This is in preparation for the next wpa update from w1.fi.

Reviewed by: philip
Tested by: philip
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D30372

src.libnames.mk: Remove redundant am-utils library definition

r358821 (13f7dbe822d5f447c6137dc0d4ff838fb7050dbe) retired amd(8).
This commit cleans up an artifact left in src.libnames.mk.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D29914

pfctl: Move ioctl abstraction functions into libpfctl

Introduce a library to wrap the pf ioctl interface.

MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29562

Remove the MK_LIBCPLUSPLUS option

This option has been equivalent to any form of C++ support since libstdc++
was removed. Therefore, replace all MK_LIBCPLUSPLUS uses with MK_CXX.

Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D27974

ncurses: only keep the version with widechar support

Only keep the widechar version of ncurses as libncursesw.so.9

Keep the old name to avoid breaking the ABI compatibility (the non
widechar version libncurses.so.9 is not binary compatible with
libncursesw.so.9) since all ports and base are already only linking
against the widechar version we can simply remove libncurses.so.9

Since the .9 version only lived in the dev branch and never ended in a
release, it is simply removed and not added to any binary compat
package.

Add symlinks to keep build time compatibility for anyone linking against
-lncurses

build: remove the option to build gnugrep

Unconditionally install bsdgrep as grep, bootstrap or not. Remove all
build glue and stop installing both gnugrep and libgnuregex now that
all consumers of the latter are gone.

Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D27732

Add virtio-9p (aka VirtFS) filesystem sharing to bhyve.

VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete support
for 9P2000.L protocol, except for the extended attribute support. It has
been verified to work with the qemu-kvm hypervisor.

Reviewed by: rgrimes, emaste, jhb, trasz
Approved by: trasz (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: Conclusive Engineering (development), vStack.com (funding)
Differential Revision: https://reviews.freebsd.org/D10335

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

libifconfig now depends on libm due to usage of log10().

ld.bfd in particular requires -lm to come after libifconfig on the
command line when linking rescue.

Reviewed by: freqlabs, adrian
Differential Revision: https://reviews.freebsd.org/D26258

lib: add libnetmap

This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCTRL) or the simple header-only-library
netmap_user.h (e.g. nm_open(), nm_close(), nm_mmap() etc.)

The new library offers more functionalities than netmap_user.h:
- Support for complex netmap options, such as external memory
allocators or per-buffer offsets. This opens the way to future
extensions.
- More flexibility in the netmap port bind options, such as
non-numeric names for pipes, or the ability to specify the netmap
allocator that must be used for a given port.
- Automatic tracking of the netmap memory regions in use across the
open ports.

At the moment there is no man page, but the libnetmap.h header file
has in-depth documentation.

Reviewed by: hrs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26171

libzfs: Add missing crypto dependency

libzfs_crypto.c uses PKCS5_PBKDF2_HMAC_SHA1 from libcrypto.

Reported by: John Kennedy
Sponsored by: iXsystems, Inc.

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

libcasper: Introduce cap_net a network service for Casper.

Reviewed by: emaste, markj (previous version), bcr (man page)
Differential Revision: https://reviews.freebsd.org/D24688

gtest: link against libregex for GNU extensions

gtest tests want to use \w ([[:alnum:]]) at the very least, which was
causing them to fail after r363679.

Start linking against libregex so that this shorthand is implemented.

PR: 248452

liblzma: Make liblzma use libmd implementation of SHA256.

MFC after: 2 weeks
PR: 200142

rename in-tree libevent v1 to libevent1

r316063 installed pf's embedded libevent as a private lib, with headers
in /usr/include/private/event. Unfortunately we also have a copy of
libevent v2 included in ntp, which needed to be updated for compatibility
with OpenSSL 1.1.

As unadorned 'libevent' generally refers to libevent v2, be explicit that
this one is libevent v1.

Reviewed by: vangyzen (earlier)
Differential Revision: https://reviews.freebsd.org/D17275

libauditd: make it a PRIVATELIB

According to the upstream man page (which we don't install), none of
libauditd's symbols are intended to be public. Also, I can't find any
evidence for a port that uses libauditd. Therefore, we should treat it like
other such libraries and use PRIVATELIB.

Reported by: phk
Reviewed by: cem, emaste
MFC after: 2 weeks

Do not claim libbearssl et al are INTERNALLIB

If INTERNALLIB is defined we need PIE and bsd.incs.mk is
not included.

PR: 245189
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org//D24233

Improve LIBADD/_DP_* for kyua libraries.

This fixes build with ld.bfd as the linker (e.g. on powerpc).

This corrects a bug in D24103.

Sponsored by: DARPA

Add an internal liblua and use it in flua.

The new liblua will be used in a forthcoming import of kyua.

Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24090

remove GCC 4.2.1 build infrastructure

As described in Warner's email message[1] to the FreeBSD-arch mailing
list we have reached GCC 4.2.1's retirement date. At this time all
supported architectures either use in-tree Clang, or rely on external
toolchain (i.e., a contemporary GCC version from ports).

GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later
that year, in r171825. GCC has served us well, but version 4.2.1 is
obsolete and not used by default on any architecture in FreeBSD. It
does not support modern C and does not support arm64 or RISC-V.

Thanks to everyone responsible for maintaining, updating, and testing
GCC in the FreeBSD base system over the years.

So long, and thanks for all the fish.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR: 228919
Reviewed by: brooks, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23124

libssp_nonshared: use only on i386 and ppc

libssp_nonshared.a defines one symbol, __stack_chk_fail_local. This
is used only on i386 and powerpc; other archs emit calls directly to
__stack_chk_fail. Simplify linking on other archs by omitting it.

PR: 242941 [exp-run]

Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943

Allow OBJDIR to be overridden for LIB*DIR variables.

This will allow us to link against internal libraries when building
programs for the system's LIBCOMPAT ABI.

Reviewed by: bdrewery
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL

Introduce stats(3), a flexible statistics gathering API.

This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof). Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs. The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by: sef (earlier version)
Obtained from: Netflix
Relnotes: yes
Sponsored by: Klara Inc, Netflix
Differential Revision: https://reviews.freebsd.org/D20477

Add the missing bits for LIBADD to properly function now that
libarchive is linked to libzstd

Pointy hat: bapt
Reported by: antoine

Remove cap_random(3).

Now that we have a way to obtain entropy in capability mode
(getrandom(2)), libcap_random is obsolete. Remove it.

Bump __FreeBSD_version in case anything happens to use it, though I've
found no consumers.

Reviewed by: delphij, emaste, oshogbo
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21033

Remove NAND and NANDFS support

NANDFS has been broken for years. Remove it. The NAND drivers that
remain are for ancient parts that are no longer relevant. They are
polled, have terrible performance and just for ancient arm
hardware. NAND parts have evolved significantly from this early work
and little to none of it would be relevant should someone need to
update to support raw nand. This code has been off by default for
years and has violated the vnode protocol leading to panics since it
was committed.

Numerous posts to arch@ and other locations have found no actual users
for this software.

Relnotes: Yes
No Objection From: arch@
Differential Revision: https://reviews.freebsd.org/D20745

Enable build of libbearssl

Reviewed by: emaste
Sponsored by: Juniper Networks
Differential Revision: D16337

Make libifconfig INTERNALLIB

Instead of PRIVATELIB + NO_PIC. This avoids the need for the wlandebug
PIE special case added in r344211, and provides a stronger guarantee
against 3rd party software coming to depend on the API or ABI.

If / when we declare the API/ABI to be stable we can make it a normal
library.

Discussed with: bapt
Sponsored by: The FreeBSD Foundation

Fix typos

LIBGMOCK_MAIN and LIBGTEST_MAIN were missing corresponding underscores in the
variable names.

Use make's :tl instead of checking "no" and "NO"

Suggested by: kevans
Reviewed by: kevans

Add WITH_PIE knob to build Position Independent Executables

Building binaries as PIE allows the executable itself to be loaded at a
random address when ASLR is enabled (not just its shared libraries).

With this change PIE objects have a .pieo extension and INTERNALLIB
libraries libXXX_pie.a.

MK_PIE is disabled for some kerberos5 tools, Clang, and Subversion, as
they explicitly reference .a libraries in their Makefiles. These can
be addressed on an individual basis later. MK_PIE is also disabled for
rtld-elf because it is already position-independent using bespoke
Makefile rules.

Currently only dynamically linked binaries will be built as PIE.

Discussed with: dim
Reviewed by: kib
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18423

Merge build glue for libraries and tests done on github

I need to doublecheck my work vs the port, but I believe that this covers the
initial integration of all upstream tests.

Ref: https://github.com/ngie-eign/freebsd/tree/googletest-integration

libcasper: introduce cap_fileargs service

cap_fileargs is a Casper service which helps to sandbox applications that need
access to the filesystem namespace. The main purpose of the service is to make
easy to capsicumize applications that works on multiple files passed in argv.

We have a couple example of using it but we still treat this service as an
experimental one.

Reviewed by: emsate (previous version), jonathan (partially)
Discussed with: many
Differential Revision: https://reviews.freebsd.org/D14407

Add libssl to libldns for DANE.

Approved by: re (gjb)

Link libcrypto with pthread.

libbe(3): Move build goop back out of cddl/

Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.

Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.

IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.

libbe(3): Add to cddl build, adjust src.libnames.mk as needed

src.libnames.mk: Make libbe entry consistent with others

Perhaps using libbe.a from "${DESTDIR}${LIBDIR}" might have been the
prevailing technique at one point, but nowadays it appears to be to
preferred to spell this as "${OBJTOP}/lib" -- make it so.

Import libbe(3)/be(1) from socsvn/soc2017/kneitinger/libbe-head

Rework ofed build.

Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from: bdrewery
Reviewed by: bdrewery, hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D15648

Created static libc PIC/no-SSP library to be used by rtld.

Rtld is not compatible with SSP, and since we link libc_pic.a to rtld
to have the basic support like memory and string copy functions, we
have to both carefully limit libc use, and to provide the ssp support
shims. This change makes the libc use in rtld more straighforward but
still limited, and allows to remove the shims, to be done in the next
commit.

Submitted by: Luis Pires
Reviewed by: bdrewery, brooks
Differential revision: https://reviews.freebsd.org/D15283

Add new shared library -- libopencsd.

OpenCSD is an ARM CoreSight(tm) trace packets decoder.

- Connect libopencsd to the arm64 build.
- Install opencsd headers to /usr/include/opencsd/

Sponsored by: DARPA, AFRL

Add new shared library -- libipt.

libipt is the Intel Processor Trace (Intel PT) packets decoder.

- Include libipt to amd64 build.
- Install libipt headers to /usr/include/libipt/

Sponsored by: DARPA, AFRL

Remove libreadline from the source tree, all consumers but gdb
has been switched to libedit long ago, libreadline was built as an
internallib for a while and kept only for gdbtui which was broken using
libreadline.

Since gdb has been mostly deorbitted in all arches, gdbtui was only installed
on arm and sparc64, given it has been removed, gdb has been switched to use
libedit, no consumers are left for libreadline. Thus this removal

Add libregex, connect it to the build

libregex is a regex(3) implementation intended to feature GNU extensions and
any other non-POSIX compliant extensions that are deemed worthy.

These extensions are separated out into a separate library for the sake of
not cluttering up libc further with them as well as not deteriorating the
speed (or lack thereof) of the libc implementation.

libregex is implemented as a build of the libc implementation with LIBREGEX
defined to distinguish this from a libc build. The reasons for
implementation like this are two-fold:

1.) Maintenance- This reduces the overhead induced by adding yet another
regex implementation to base.

2.) Ease of use- Flipping on GNU extensions will be as simple as linking
against libregex, and POSIX-compliant compilations can be guaranteed with a
REG_POSIX cflag that should be ignored by libc/regex and disables extensions
in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when
implemented in this fashion.

Tests are added for future functionality, but left disconnected for the time
being while other testing is done.

Reviewed by: cem (previous version)
Differential Revision: https://reviews.freebsd.org/D12934

Look for libraries in OBJDIR when building outside of buildworld.

This allows build testing more easily without establishing a sysroot
or installing the files.

Sponsored by: Dell EMC

Fix OFED library dependencies.

Sponsored by: Mellanox Technologies

Remove no longer supported mthca driver.

Sponsored by: Mellanox Technologies

Introduce syslog service for Casper.

syslog in libc secretly reconnects to the daemon.
Another issue is that we don't have any information from openlog(3) if we
succeeded to open log or not so we don't know if we are ready
to enter cabability mode.
Because all of that we decided we need a syslog service for Caspser.

Reviewed by: bapt@
Differential Revision: https://reviews.freebsd.org/D12824

Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12718

Mark libifconfig as private library in src.libnames.mk (completes r305700)

Provide libdl.

Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.

Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.

Reviewed by: bdrewery, dim (previos version); emaste
Exp run: PR 220525, done by antoine
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D11504

Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.

Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
A LIBSTAND hack is no longer required for buildenv.

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Fix Makefiles which override LIBDIR to not add incorrect dependencies into .depend.

This fixes these cases which would rebuild every time:
make[6]: /usr/obj/usr/src/libexec/rtld-elf/tests/libpythagoras/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/libexec/rtld-elf/libm.a
make[6]: /usr/obj/usr/src/lib/libxo/tests/encoder/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libxo/libxo.a
make[7]: /usr/obj/usr/src/lib/libthr/tests/dlopen/dso/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libthr/dlopen/libpthread.a

The problem is that some Makefiles will override LIBDIR to where they want
their library to install. bsd.libnames.mk will then use ${LIBDIR} to define
where *existing* libraries are. This then leads to looking for the
libraries in the *target* place rather than the *expected* place.

We may want to expand this (and all of the other *DIR variables in bsd.own.mk)
into something like what Ports has, a PREFIX and a LOCALBASE. PREFIX being
where things are being installed to and LOCALBASE being where they already are.

For now store the default expected LIBDIR into LIBDIR_BASE and use that for
library locations.

Reported by: sbruno
MFC after: 1 week
Sponsored by: Dell EMC Isilon

Fix more incorrect library directories fix 'stale .depend' rebuilds.

Reported by: sbruno
MFC after: 3 days
Sponsored by: Dell EMC Isilon

Fix LIBAMU location to fix 'stale .depend' rebuilds in usr.sbin/amd.

This originally came in r275052.

Reported by: sbruno
MFC after: 3 days
Sponsored by: Dell EMC Isilon

Build zstandard with threading enabled

Create libefivar library. This library aims to provide
the same API as the GPL'd version of this library. It implements the common
Linux API for programatically manipulating UEFI environment varibales using
the UEFI Runtime Services the kernel provides. It replaces the old efi
library since it is programmed to a different interface, but retails the
CHAR16 to UTF-8 and vice versa conversion routines. The new name is to match
Linux program's expectations.

Differential Revision: https://reviews.freebsd.org/D8128
Reviewed by: kib@, wblock@, Ganael Laplanche

Remove cruft that accidently crept in r305931

Disable GNU rcs by default

All remaining tools using rcs has been switched to directly use diff3(1):
- etcupdate(8)
- freebsd-update(8)

Note that the ident(1) tool is been already replaced long ago with a BSD
licensed version, as such it remains installed.

GNU rcs is still available from ports:
- rcs: newer GPLv3 version
- rcs57: the latest version from base (GPLv2)

Renaming libifc to libifconfig in response to feedback on initial commit of
this library. Sticking to 'libifconfig' (and 'ifconfig_' as function prefix)
should reduce chances of namespace collisions, make it more clear what the
library does, and be more in line with existing libraries.

Submitted by: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Differential Revision: https://reviews.freebsd.org/D7742
Reviewed by: cem, kp

Add libifc, a library implementing core functionality that exists in ifconfig(8) today.

libifc (pronounced lib-ifconfig) aims to be a light abstraction layer between
programs and the kernel APIs for managing the network configuration.
This should hopefully make programs easier to maintain, and reduce code
duplication.

Work will begin on making ifconfig(8) use this library in the near future.

This code is still evolving. The interface should not be considered stable until
it is announced as such.

Submitted By: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D7529

libproc: Add proc_getmodel().

This is used by libdtrace to determine the data model of target processes.
This allows for the creation of pid provider probes in 32-bit processes on
amd64.

MFC after: 1 month

librtld_db: Use the auxv to figure out where to look up loader symbols.

Previously, librtld_db just hardcoded /libexec/ld-elf.so, which isn't
correct for processes that aren't using the native ABI. With this change,
librtld_db can be used to inspect non-native processes; in particular,
dtrace -c now works for 32-bit executables on amd64.

MFC after: 1 month

Add basic blacklist build support

Reviewed by: rpaulo
Approved by: rpaulo
Relnotes: YES
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5913

Fix the library name.

Reviewed by: bdrewery

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

Fix over/under-linking in contrib/ofed.

Sponsored by: EMC / Isilon Storage Division

Add missing atf dirs.

Sponsored by: EMC / Isilon Storage Division

Move casper library entries to proper places.

Sponsored by: EMC / Isilon Storage Division

Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277

Add libpe for elfcopy(1) PE/COFF support

Sponsored by: The FreeBSD Foundation

Add comment about OBJTOP not yet being defined outside of META MODE.

It is fine for the purpose of the check though.

Sponsored by: EMC / Isilon Storage Division

Follow-up r291739: Don't suggest LDADD on private libs to use LIBADD.

This is because LDADD+=-lFOO is not the same as LDADD+=-lprivateFOO which is
what the private libs in LIBADD are.

Sponsored by: EMC / Isilon Storage Division

Start on a new library (libsysdecode) that provides routines for decoding
system call information such as system call arguments. Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.

This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it. One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events. This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events. Instead, this function only generates a string
description for known malloc() and RTLD records.

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D4537

Let termcap be used in LIBADD.

Sponsored by: EMC / Isilon Storage Division

Convert contrib/ofed libraries to LIBADD.

Sponsored by: EMC / Isilon Storage Division

Sort the LIB list.

Sponsored by: EMC / Isilon Storage Division

Add assertion for when LIBADD should be used rather than LDADD/DPADD.

Sponsored by: EMC / Isilon Storage Division

Rework unknown LIBADD assertion to be more clear and to not suggest adding
DPADD/LDADD_<foo> variables that are a special case.

Sponsored by: EMC / Isilon Storage Division

Support all of the CDDL/ZFS libraries for LIBADD.

Sponsored by: EMC / Isilon Storage Division

For INTERNALLIB always add in the corresponding _DP_ and use LIBADD in
the real build file.

This lessens the need to define DPADD_<lib> and LDADD_<lib> to just very
special cases.

Sponsored by: EMC / Isilon Storage Division

Fix a typo in a comment (spacial -> special)

Sponsored by: EMC / Isilon Storage Division

Add assertions that capture invalid configurations for new libraries.

Fix current findings, which should fix cases of NO_SHARED not building
properly.

Given libfoo:
- Ensure that a LIBFOO is set. For INTERNALLIBS advise setting this in
src.libnames.mk, otherwise bsd.libnames.mk.
- Ensure that a LIBFOODIR is properly set.
- Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
Makefile

Sponsored by: EMC / Isilon Storage Division

Don't overlink pthread to kerberos library consumers.

I'm not sure why this was here, none of these use pthread themselves and
none of the consumers are broken with removing this.

Sponsored by: EMC / Isilon Storage Division

Don't overlink pthread to libsqlite3 consumers.

At least usr.bin/mandoc was overlink.

Sponsored by: EMC / Isilon Storage Division

Fix truncation of _DP_proc and add missing libelf.

Sponsored by: EMC / Isilon Storage Division

Reduce overlinking of libdtrace consumers with libctf, libelf, libproc.

The proper place for this list is _DP_dtrace.

Due to removing the LDADD_dtrace, more LIBADD are needed in
cddl/usr.sbin/dtrace to prevent underlinking.

This fixes overlinking in cddl/usr.sbin/lockstat and
cddl/usr.sbin/plockstat.

Sponsored by: EMC / Isilon Storage Division