libnvmf: Add internal library to support NVMe over Fabrics

libnvmf provides APIs for transmitting and receiving Command and
Response capsules along with data associated with NVMe commands.
Capsules are represented by 'struct nvmf_capsule' objects.

Capsules are transmitted and received on queue pairs represented by
'struct nvmf_qpair' objects.

Queue pairs belong to an association represented by a 'struct
nvmf_association' object.

libnvmf provides additional helper APIs to assist with constructing
command capsules for a host, response capsules for a controller,
connecting queue pairs to a remote controller and optionally
offloading connected queues to an in-kernel host, accepting queue pair
connections from remote hosts and optionally offloading connected
queues to an in-kernel controller, constructing controller data
structures for local controllers, etc.

libnvmf also includes an internal transport abstraction as well as an
implementation of a userspace TCP transport.

libnvmf is primarily intended for ease of use and low-traffic use cases
such as establishing connections that are handed off to the kernel.
As such, it uses a simple API built on blocking I/O.

For a host, a consumer first populates an 'struct
nvmf_association_params' with a set of parameters shared by all queue
pairs for a single association such as whether or not to use SQ flow
control and header and data digests and creates a 'struct
nvmf_association' object. The consumer is responsible for
establishing a TCP socket for each queue pair. This socket is
included in the 'struct nvmf_qpair_params' passed to 'nvmf_connect' to
complete transport-specific negotiation, send a Fabrics Connect
command, and wait for the Connect reply. Upon success, a new 'struct
nvmf_qpair' object is returned. This queue pair can then be used to
send and receive capsules. A command capsule is allocated, populated
with an SQE and optional data buffer, and transmitted via
nvmf_host_transmit_command. The consumer can then wait for a reply
via nvmf_host_wait_for_response. The library also provides some
wrapper functions such as nvmf_read_property and nvmf_write_property
which send a command and wait for a response synchronously.

For a controller, a consumer uses a single association for a set of
incoming connections. A consumer can choose to use multiple
associations (e.g. a separate association for connections to a
discovery controller listening on a different port than I/O
controllers). The consumer is responsible for accepting TCP sockets
directly, but once a socket has been accepted it is passed to
nvmf_accept to perform transport-specific negotiation and wait for the
Connect command. Similar to nvmf_connect, nvmf_accept returns a newly
construct nvmf_qpair. However, in contrast to nvmf_connect,
nvmf_accept does not complete the Fabrics negotiation. The consumer
must explicitly send a response capsule before waiting for additional
command capsules to arrive. In particular, in the kernel offload
case, the Connect command and data are provided to the kernel
controller and the Connect response capsule is sent by the kernel once
it is ready to handle the new queue pair.

For userspace controller command handling, the consumer uses
nvmf_controller_receive_capsule to wait for a command capsule.
nvmf_receive_controller_data is used to retrieve any data from a
command (e.g. the data for a WRITE command). It can be called
multiple times to split the data transfer into smaller sizes.
nvmf_send_controller_data is used to send data to a remote host in
response to a command. It also sends a response capsule indicating
success, or an error if an internal error occurs. nvmf_send_response
is used to send a response without associated data. There are also
several convenience wrappers such as nvmf_send_success and
nvmf_send_generic_error.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44710

Skip building libclang_rt when WITHOUT_CLANG is used

As noted in bug 277096, when building a pkgbase repository using
WITHOUT_CROSS_COMPILER and WITHOUT_TOOLCHAIN (which sets WITHOUT_CLANG),
the following residual files are left over:

/usr/lib/clang/18/lib/freebsd/libclang_rt.asan-x86_64.so
/usr/lib/clang/18/share/asan_ignore_list.txt
/usr/lib/clang/18/share/cfi_ignore_list.txt
/usr/lib/clang/18/share/msan_ignore_list.txt

This is because the lib/libclang_rt directory is still descended into,
even if WITHOUT_CLANG is used. Fix it by not descending into the
libclang_rt directory in that case.

PR: 277096
Reported by: Siva Mahadevan <me@svmhdvn.name>
MFC after: 3 days

arm64: Connect bhyve and libvmmapi to the build

Reviewed by: corvink, andrew, jhb, emaste
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41742

diff: Integrate libdiff from OpenBSD GoT.

This adds support for two new diff algorithms, Myers diff and Patience
diff.

These algorithms perform a different form of search compared to the
classic Stone algorithm and support escapes when worst case scenarios
are encountered.

Add the -A flag to allow selection of the algorithm, but default to
using the new Myers diff implementation.

The libdiff implementation currently only supports a subset of input and
output options supported by diff. When these options are used, but the
algorithm is not selected, automatically fallback to the classic Stone
algorithm until support for these modes can be added.

Based on work originally done by thj@ with contributions from kevans@.

Sponsored by: Klara, Inc.
Reviewed by: thj
Differential Revision: https://reviews.freebsd.org/D44302

libfdt: Make an internal FDT library available

This will be used by bhyve to build a device tree when booting arm64
guests.

Reviewed by: corvink, jhb
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40994

libc: link libsys as a auxiliary filter library

At runtime, when rtld loads libc it will also load libsys. For each
symbol that is present in both, the libsys one will override the libc
one. It continues to be the case that program need only link against
libc (usually implicitly). The linkage to libsys is automatic.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908

libsys: plumb in to build

libsys provides the FreeBSD kernel interface (auxargs, system calls,
vdso). It can be linked directly for programs using a non-standard
libc and will later be linked as a filter library to libc providing
the actual system call implementation.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908

lib: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

bsddialog: Import version 1.0

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

arm64 lib32: enable building of lib32 on arm64

Enable LIB32 option on aarch64, defaulting to YES; it had defaulted
to "broken". Add required variables for how to compile lib32 on
arm. Use /usr/include/arm for armv7 (32-bit) headers, analogous to
/usr/include/i386 on amd64. Omit libomp from lib32; it is not
supported on armv7.

Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D40945

lib: Migrate from COMPAT_32BIT to generic COMPAT_LIBCOMPAT

In the process, delete a COMPAT_SOFTFP remnant that was missed in
previous sweeps.

See commit 8fad2cda93c7 ("bsd.compat.mk: Provide new CPP and sub-make
variables") for the context behind this change.

Reviewed by: emaste, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40931

lib: Enable libcbor and libfido2 for COMPAT_32BIT

We no longer have COMPAT_32BIT hacks for libusb, instead supporting the
normal 32-bit ioctls for freebsd32 processes, so we can enable these for
the lib32 build.

Reviewed by: emaste, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40920

Add nss_tacplus, a TACACS+ NSS module.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D40133

NgATM: Remove netgraph ATM support

Most ATM support was removed prior to FreeBSD 12. The netgraph support
was kept as it was less intrusive, but it is presumed to be unused.

Reviewed by: manu
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D38879

Import zstd 1.5.2

Retire WITHOUT_CXX option

Several important base system components are written in C++, and the
WITHOUT_CXX option produced a system that was not fully functional.
Just accept this, and remove the option to build without C++ support.

This reverts commit adc3c128c6603054586a993d117e5dd808deac17.

Reviewed by: brooks, kevans, jhb (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33108

Put OPIE to rest.

Differential Revision: https://reviews.freebsd.org/D36592

contrib/bsddialog: Import version 0.4

Improvements and changes to integrate bsddialog(1) with scripts in BASE.
Overview:

* New options. --and-widget, --keep-tite, --calendar.
* Change output format. Menus and --print-maxsize.
* Redefine sizing. Fixed rows, cols and menurows became at the most.
* Add DIAGNOSTICS. Error messages for bad arguments and options.
* Add keys. Space for --menu, fast keys for --msgbox and --yesno.
* Text. Change default text modification, add --cr-wrap.

See /usr/src/contrib/bsddialog/CHANGELOG '2022-09-24 Version 0.4'
for more detailed information.

contrib/bsddialog: Import version 0.3

New features overview:
* Unicode. User interface handles multi-column characters. API can
handle char* like a multibyte character string. Internally wide
characters are used for keyboard input, to adapt word wrapping and
dynamic text auto-sizing for multi-column characters.
* Forms refactoring. Complete rewrite deleting libformw dependency.
* Theme. New utility options to save and load custom theme at run-time.
* TUI navigation. Added keys to navigate input components. Changed
default focus behavior of input dialogs to be LGPL-dialog-like; a new
option can set the previous whiptail-like behavior.

See /usr/src/contrib/bsddialog/CHANGELOG '2022-08-29 Version 0.3'
for more detailed information.

lib: Gate libfigpar under MK_DIALOG

libfigpar is only used by dpv and dpv isn't built if WITHOUT_DIALOG
is set.

Simplify conditional guards for libclang_rt.

All supported architectures can build libclang_rt now.

Reviewed by: dim
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D35740

riscv: actually enable sanitizers

1. Backport https://github.com/llvm/llvm-project/commit/b475ce39e8b1de3a70ea242473f136a567be46e3.
2. Enable libclang_rt for riscv.

Previous commit missed it.

MFC after: 3 days
Reviewed by: dim
Differential Revision: https://reviews.freebsd.org/D34543

bsddialog: Import version0.2

This version provides improvements and fixes mainly to use bsddialog
utility in bsdinstall/scripts. The lib API is not broken so the
previous converted utilities (tzsetup, distextract, etc.) are OK.

Approved by: bapt (mentor)
Differential Revision: https://reviews.freebsd.org/D34458

bsddialog: import version 0.1

Remove quotes around Makefile .error/.warn/.info strings

The text after .error et al is emitted verbatim.

Reviewed by: sjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33904

Add an internal libiscsiutil library.

Move some of the code duplicated between ctld(8) and iscsid(8) into a
libiscsiutil library.

Sharing the low-level PDU code did require having a
'struct connection' base class with a method table to permit separate
initiator vs target behavior (e.g. in handling proxy PDUs).

Reviewed by: mav, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33544

bsddialog: import snapshot 0.0.2

bsddialog: import version 0.0.1

Apply ASAN/UBSAN Makefile check only for build target

COMPILER_TYPE is not set during cleandir and perhaps other non-build
targets, and a build with ASAN or UBSAN enabled failed with an error
reporting that runtime libraries could not be built.

PR: 260099
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32805

bsddialog: vendor import 2021-12-05

bsddialog, mark the lib as dependent on ncurses

bsddialog: import new TUI widget and utility

bsddialog is an attempt to write in permissive license a replacement for
libdialog.

While it is still in early stage it is good enough to already be used in
many areas, it is imported as private lib until it matures enough to be
considered as having a stable ABI

bsddialog: import snapshot as of 2021-11-24

Don't build sanitizer runtimes under WITHOUT_CXX

In the past we built the sanitizer runtimes when building Clang
(and using Clang as the compiler) but 7676b388adbc changed this to
be conditional only on using Clang, to make the runtimes available
for external Clang.

They fail to build when WITHOUT_CXX is set though, so add MK_CXX
as part of the condition.

Reported by: Michael Dexter, Build Option Survey
Reviewed by: imp, jrtc27
Fixes: 7676b388adbc ("Always build the sanitizer runtimes...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32731

Avoid building libfido2 if WITHOUT_USB

libfido2 requires USB, so disable it if not available.

Reported by: peterj
Fixes: 7b1e19ad78c6 ("Add libfido2 to the build")
Sponsored by: The FreeBSD Foundation

Add libfido2 to the build

From https://github.com/Yubico/libfido2:

libfido2 provides library functionality and command-line tools to
communicate with a FIDO device over USB, and to verify attestation
and assertion signatures.

libfido2 supports the FIDO U2F (CTAP 1) and FIDO 2.0 (CTAP 2)
protocols.

libfido2 will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32448

Add libcbor to the build

From https://github.com/PJK/libcbor:

libcbor is a C library for parsing and generating CBOR, the general-
purpose schema-less binary data format.

libcbor will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

cbor_export.h and configuration.h were generated by the upstream CMake
build. We could create them with bmake rules instead (as NetBSD has
done) but this is a fine start.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h, and there is
no need for libcbor without libfido2.

Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32347

Implement and use new mixer(3) library for FreeBSD.

Wiki article: https://wiki.freebsd.org/SummerOfCode2021Projects/SoundMixerImprovements
This project was part of Google Summer of Code 2021.

Submitted by: christos@
Differential Revision: https://reviews.freebsd.org/D31636
Sponsored by: NVIDIA Networking

Add WITH_LLVM_BINUTILS to install LLVM binutils instead of Elftoolchain

When WITH_LLVM_BINUTILS is set, we will install the LLVM binutils as
ar/ranlib/nm/objcopy/etc. instead of the elftoolchain ones.
Having the LLVM binutils instead of the elftoolchain ones allows us to use
features such as LTO that depend on binutils that understand LLVM IR.
Another benefit will be an improved user-experience when compiling with
AddressSanitizer, since ASAN does not symbolize backtraces correctly if
addr2line is elftoolchain addr2line instead of llvm-symbolizer.
See https://lists.freebsd.org/archives/freebsd-toolchain/2021-July/000062.html
for more details.

This is currently off by default but will be turned on by default at some
point in the near future.

Reviewed By: emaste

Differential Revision: https://reviews.freebsd.org/D31060

Add build system support for ASAN+UBSAN instrumentation

This adds two new options WITH_ASAN/WITH_UBSAN that can be set to
enable instrumentation of all binaries with AddressSanitizer and/or
UndefinedBehaviourSanitizer. This current patch is almost sufficient
to get a complete buildworld with sanitizer instrumentation but in
order to actually build and boot a system it depends on a few more
follow-up commits.

Reviewed By: brooks, kib, markj
Differential Revision: https://reviews.freebsd.org/D31043

Remove lib/kyua from the build

I forgot to include this line in 2eb9ad427475190e87d026de4ca7fc296947f2a0.

Reported by: Jenkins CI
MFC after: 1 week
Fixes: 2eb9ad427475190ei ("Simplify and speed up the kyua build")

pfctl: Move ioctl abstraction functions into libpfctl

Introduce a library to wrap the pf ioctl interface.

MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29562

Always build the sanitizer runtimes when compiling with clang

This allows instrumenting e.g. test binaries even when compiling with an
external clang (e.g. CROSS_TOOLCHAIN=llvm11). I have some upcoming patches
that allow building the entire base system with ASan/UBSan/etc.
instrumentation and this is required in preparation for this.

Reviewed By: dim, emaste
Differential Revision: https://reviews.freebsd.org/D28532

Remove the MK_LIBCPLUSPLUS option

This option has been equivalent to any form of C++ support since libstdc++
was removed. Therefore, replace all MK_LIBCPLUSPLUS uses with MK_CXX.

Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D27974

build: remove LIBPTHREAD/LIBTHR build options

WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building
a useful system without threading support is becoming increasingly more
difficult.

Additionally, in the five plus years that it's been broken more reverse
dependencies have cropped up in libzstd, libsqlite3, and libcrypto (among
others) that make it more and more difficult to reconcile the effort needed
to fix these options.

Remove the broken options.

PR: 252760
Reviewed by: brooks, emaste, kib
Differential Revision: https://reviews.freebsd.org/D28263

Import zstd 1.4.8

flua: Add a libjail module

libjail is pretty small, so it makes for a good proof of concept demonstrating
how a system library can be wrapped to create a loadable Lua module for flua.

* Introduce 3lua section for man pages
* Add libjail module

Reviewed by: kevans, manpages
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D26080

Add virtio-9p (aka VirtFS) filesystem sharing to bhyve.

VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete support
for 9P2000.L protocol, except for the extended attribute support. It has
been verified to work with the qemu-kvm hypervisor.

Reviewed by: rgrimes, emaste, jhb, trasz
Approved by: trasz (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: Conclusive Engineering (development), vStack.com (funding)
Differential Revision: https://reviews.freebsd.org/D10335

lib: add libnetmap

This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCTRL) or the simple header-only-library
netmap_user.h (e.g. nm_open(), nm_close(), nm_mmap() etc.)

The new library offers more functionalities than netmap_user.h:
- Support for complex netmap options, such as external memory
allocators or per-buffer offsets. This opens the way to future
extensions.
- More flexibility in the netmap port bind options, such as
non-numeric names for pipes, or the ability to specify the netmap
allocator that must be used for a given port.
- Automatic tracking of the netmap memory regions in use across the
open ports.

At the moment there is no man page, but the libnetmap.h header file
has in-depth documentation.

Reviewed by: hrs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26171

Move libsqlite3 to the top of the SUBDIR list

In parallel builds, this should allow sqlite to start building earlier and
increase parallelism when building lib/. Looking at htop output during
buildworld/tinderbox, there are long phases where only one CPU is active
optimizing the massive sqlite3.c file since the build of libsqlite3 is
started quite late.

Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D26169

Ensure libregex is built in time for googletest

In lib/Makefile, we document the dependency with SUBDIR_DEPEND

For buildworld orchestration, just prebuild libregex if GOOGLETEST is
enabled. googletest will get built in a later pass.

rename in-tree libevent v1 to libevent1

r316063 installed pf's embedded libevent as a private lib, with headers
in /usr/include/private/event. Unfortunately we also have a copy of
libevent v2 included in ntp, which needed to be updated for compatibility
with OpenSSL 1.1.

As unadorned 'libevent' generally refers to libevent v2, be explicit that
this one is libevent v1.

Reviewed by: vangyzen (earlier)
Differential Revision: https://reviews.freebsd.org/D17275

Import Zstd 1.4.5

Add an internal liblua and use it in flua.

The new liblua will be used in a forthcoming import of kyua.

Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24090

retire the LLVM_LIBUNWIND option

LLVM's libunwind is used on all FreeBSD-supported CPU architectures and
is a required component.

Reviewed by: brooks (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23123

Remove excluding libproc and librtld_db on sparc64.

Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943

libclang_rt: enable on powerpc*

Summary:
Enable on powerpc64 and in lib/libclang_rt/Makefile change
MACHINE_CPUARCH to MACHINE_ARCH because on powerpc64
MACHINE_ARCH==MACHINE_CPUARCH so the 32-bit library overwrites 64-bit
library during installworld.

This patch doesn't enable any other libclang_rt libraries because they
need to be separately ported.

I have verified that games/julius (which fails on powerpc64 elfv2
without this change because of no libclang_rt profiling library) builds.

Test Plan: Ship it, test on powerpc and powerpcspe

Submitted by: pkubaj
Reviewed by: dim, jhibbits
Differential Revision: https://reviews.freebsd.org/D22425
MFC after: 1 month
X-MFC-With: r353358

Vendor import of NetBSD's external/bsd/blacklist @ 20191106

Sponsored by: The FreeBSD Foundation

Import Zstd 1.4.4

Introduce stats(3), a flexible statistics gathering API.

This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof). Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs. The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by: sef (earlier version)
Obtained from: Netflix
Relnotes: yes
Sponsored by: Klara Inc, Netflix
Differential Revision: https://reviews.freebsd.org/D20477

Enable OpenMP for powerpc64

Summary: When powerpc64 switches to LLVM, use this patch to enable
OpenMP as well. OpenMP on PPC is only for 64-bits, so don't make a
32-bit libomp. A change to openmp files is necesssary (under review on
https://reviews.llvm.org/D67190), because it determines ELF format
version based on endianness, which is incorrect.

Reviewed by: alfredo.junior_eldorado.org.br, #manpages
Differential Revision: https://reviews.freebsd.org/D21532

Add native support for zstd to libarchive

Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected

PR: 238797
Exp run by: antoine
Reviewed by: cem
Approved by: cem
Differential Revision: https://reviews.freebsd.org/D20752

Update build glue for building libc++.

import zstd 1.4.1

Remove NAND and NANDFS support

NANDFS has been broken for years. Remove it. The NAND drivers that
remain are for ancient parts that are no longer relevant. They are
polled, have terrible performance and just for ancient arm
hardware. NAND parts have evolved significantly from this early work
and little to none of it would be relevant should someone need to
update to support raw nand. This code has been off by default for
years and has violated the vnode protocol leading to panics since it
was committed.

Numerous posts to arch@ and other locations have found no actual users
for this software.

Relnotes: Yes
No Objection From: arch@
Differential Revision: https://reviews.freebsd.org/D20745

import zstd 1.4.0

Enable building libomp.so for 32-bit x86. This is done by selectively
enabling the functions that save and restore MXCSR, since access to this
register requires SSE support.

Note that you may run into other issues with OpenMP on i386, since this
*not* yet supported upstream, and certainly not extensively tested.

PR: 236062, 236582
MFC after: 1 month
X-MFC-With: r344779

Disable lib/libomp build for the 32-bit part of amd64 buildworld, as it
is not supported for that target.

Reported by: Michael Butler <imb@protected-networks.net>
PR: 236062
MFC after: 1 month
X-MFC-With: r344779

Connect lib/libomp to the build.

* Set MK_OPENMP to yes by default only on amd64, for now.
* Bump __FreeBSD_version to signal this addition.
* Ensure gcc's conflicting omp.h is not installed if MK_OPENMP is yes.
* Update OptionalObsoleteFiles.inc to cope with the conflicting omp.h.
* Regenerate src.conf(5) with new WITH/WITHOUT fragments.

Relnotes: yes
PR: 236062
MFC after: 1 month
X-MFC-With: r344779

Enable build of libbearssl

Reviewed by: emaste
Sponsored by: Juniper Networks
Differential Revision: D16337

Add MK_GOOGLETEST option for building/installing gmock/gtest

These libraries don't compile on non-C++-11 capable compilers, e.g., g++ 4.2.1
and its corresponding implementation of the c++ library, i.e., libstdc++.

Blacklist compilation on all non-C++-11 capable compilers and give others the
option of opting out of building/installing gmock/gtest via MK_GOOGLETEST.

This option is controlled by MK_CXX and MK_TESTS, as ATF compilation is.

Merge build glue for libraries and tests done on github

I need to doublecheck my work vs the port, but I believe that this covers the
initial integration of all upstream tests.

Ref: https://github.com/ngie-eign/freebsd/tree/googletest-integration

import zstd 1.3.8

retire LINKER_FEATURES filter flag

And build libdl unconditionally. All supported FreeBSD linkers accept
-F / --filter so there is no need to test for support.

Discussed with: kib
Sponsored by: The FreeBSD Foundation

Move pmc* bits behind MK_PMC to fix WITHOUT_PMC build

No objection from: mmacy
MFC after: 3 days

import zstd 1.3.7

re-enable pmcstat, pmccontrol, and pmcannotate for gcc4 builds

I had disabled building of the aforementioned targets due to warnings breaking
tinderbox. This silences the warning and restores them to the build.

Reported by: jhibbits
Reviewed by: jhibbits
Approved by: re (gjb)

libbe(3): Move build goop back out of cddl/

Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.

Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.

IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.

Remove special cases for armeb in the build.

Differential Revision: https://reviews.freebsd.org/D16257

Normalize the g(eom,cache,part,...) build.

Rather then combining hardlink creation for the geom(8) binary with
shared library build, move libraries to src/lib/geom so they are
built and installed normally. Create a common Makefile.classes
which is included by both lib/geom/Makefile and sbin/geom/Makefile
so the symlink and libraries stay in sync.

The relocation of libraries allows libraries to be build for 32-bit
compat. This also reduces the number of non-standard builds in
the system.

This commit is not sufficent to run a 32-bit /sbin/geom on a 64-bit
system out of the box as it will look in the wrong place for libraries
unless GEOM_LIBRARY_PATH is set appropriatly in the environment.

Reviewed by: bdrewery
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15360

Rework ofed build.

Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from: bdrewery
Reviewed by: bdrewery, hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D15648

pmc: fix logic in skipping riscv

pmc: don't build on riscv where there's no kmod support

pmc: convert native to jsonl and track TSC value of samples

- add '-j' options to filter to enable converting native pmc
log format to json lines format to enable the use of scripts
and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
recording nanotime when the sample is copied to a global log
in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
us an offset for translating the tsc values in callchain records

Add new shared library -- libopencsd.

OpenCSD is an ARM CoreSight(tm) trace packets decoder.

- Connect libopencsd to the arm64 build.
- Install opencsd headers to /usr/include/opencsd/

Sponsored by: DARPA, AFRL

Add new shared library -- libipt.

libipt is the Intel Processor Trace (Intel PT) packets decoder.

- Include libipt to amd64 build.
- Install libipt headers to /usr/include/libipt/

Sponsored by: DARPA, AFRL

Add libregex, connect it to the build

libregex is a regex(3) implementation intended to feature GNU extensions and
any other non-POSIX compliant extensions that are deemed worthy.

These extensions are separated out into a separate library for the sake of
not cluttering up libc further with them as well as not deteriorating the
speed (or lack thereof) of the libc implementation.

libregex is implemented as a build of the libc implementation with LIBREGEX
defined to distinguish this from a libc build. The reasons for
implementation like this are two-fold:

1.) Maintenance- This reduces the overhead induced by adding yet another
regex implementation to base.

2.) Ease of use- Flipping on GNU extensions will be as simple as linking
against libregex, and POSIX-compliant compilations can be guaranteed with a
REG_POSIX cflag that should be ignored by libc/regex and disables extensions
in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when
implemented in this fashion.

Tests are added for future functionality, but left disconnected for the time
being while other testing is done.

Reviewed by: cem (previous version)
Differential Revision: https://reviews.freebsd.org/D12934

Install strings unconditionally

Previously it was enabled by WITH_/WITHOUT_TOOLCHAIN, but it is commonly
expected to be available and may have non-toolchain consumers. As it
is now taken from the BSD-licensed ELF Tool Chain project, just install
it unconditionally.

PR: 213665, 223725
Reviewed by: bdrewery
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8398

import zstd 1.3.2

Connect libpathconv, disconnected since import in r309035.

Sponsored by: Dell EMC Isilon

Introduce caspermocks.

The idea behinds mocks is that we don't need to ifdef a lot of code in
tools itself but those defines are hidden in the casper library.
Right now the mocks are implemented as define/inlines functions.
There was a very long discussion how this should be implemented.
This approach has some advantages like we don't need to link to any additional
libraries. Unfortunately there are also some disadvantages for example it is
easy to get library out of sync between two versions of functions or that we
need extra define to compile program with casper support.
This isn't an ideal solution but it's good enough for now and should simplify
capsicumizing programs. This also doesn't close us any other ways to do those
mocks and this should evolve in time.

Discussed with: pjd, emaste, ed, rwatson, bapt, cem, bdrewery
Differential Revision: https://reviews.freebsd.org/D8753

Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12718

After the import of libc++ 5.0.0, there is no need to disable building
libc++experimental.a on arm (r318654) and mips (r318859) anymore, since
upstream fixed the static assertions which would occur.

Noticed by: George Abdelmalik <gabdelmalik@uniridge.com.au>
PR: 223119
MFC after: 3 days

Disconnect libstand from the build.

Remove libstand from the src/lib build. Remove LIBSTAND from
bsd.libnames.mk. Add affected files to the obsolete files list.

Sponsored by: Netflix

Import zstd 1.3.0

Provide libdl.

Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.

Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.

Reviewed by: bdrewery, dim (previos version); emaste
Exp run: PR 220525, done by antoine
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D11504

[lib] disable libc++experimental on mips platforms for now.

This breaks at least mips32 platform builds.

Reviewed by: dim

For now, disable building libc++experimental for arm, since there are a
number of static assertion failures in the time_t related parts.

Reported by: mmel, kib

Add libc++experimental.a for std::experimental support

This adds a separate library for supporting std::experimental features.
It is purposefully static, and must be explicitly linked into programs
using -lc++experimental.

PLEASE NOTE: there is NO WARRANTY as to any stability or continuing
existence of the features in the std::experimental parts of the C++
library!

Reviewed by: ed
Differential Revision: https://reviews.freebsd.org/D10840

Import Zstandard 1.2.0
Among changes: threading support is now default and supports freebsd

Import zstd 1.1.4

Fix typo in lib/Makefile

The SUBDIR_DEPEND variable should be for librpcsec_gss, not
liblibrpc_gss

MFC after: 1 week
PR: 216409
Reported by: mail@fbsd.e4m.org

Sync ^/vendor/NetBSD/tests/dist with upstream

Connect new LLVM-based libgcc_eh & libgcc_s to the build

Compiler-rt and LLVM's libunwind provide a suitable replacement for
libgcc.a, libgcc_eh.a, and libgcc_s.so.

Remove the now-unused LLVM_LIBUNWIND block from gnu/lib/libgcc.

PR: 213480 [exp-run]
Reviewed by: brooks, ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8189

Simplify logic for libproc and librtld_db in lib/Makefile

partially convert lib/Makefile to SUBDIR.${MK_FOO} style

Cases other than MK_* (e.g. ${MACHINE_CPUARCH} == "i386") have been left
as is.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8246

garbage collect _libatm, missed in r179308

Fix building on i386 and arm. But 'public domain' headers on the files
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.

Submitted by: kib@, dim@, zbb@, emaste@

Create libefivar library. This library aims to provide
the same API as the GPL'd version of this library. It implements the common
Linux API for programatically manipulating UEFI environment varibales using
the UEFI Runtime Services the kernel provides. It replaces the old efi
library since it is programmed to a different interface, but retails the
CHAR16 to UTF-8 and vice versa conversion routines. The new name is to match
Linux program's expectations.

Differential Revision: https://reviews.freebsd.org/D8128
Reviewed by: kib@, wblock@, Ganael Laplanche

libcapsicum: introduce Capsicum helpers

Capsicum helpers are a set of inline functions which goal is to reduce
duplicated patterns used to Capsicumize applications.

Reviewed by: cem, AllanJude, bapt, ed, emaste
Differential Revision: https://reviews.freebsd.org/D8013

Add librss, a simple wrapper around RSS APIs so applications can begin auto-tuning.

I've used this in a handful of RSS test applications. It is just some
very simple functions to fetch the RSS configuration, query the per-bucket
CPU set, and mark sockets as local to an RSS bucket. It should be sufficient
for both thread-based and process-based workloads.

(Yes, I wrote a manpage.)

This is based on some early RSS API and wrapper API work I did whilst
I was at Netflix. Thanks to Netflix for the very original work that
spawned this; thanks to Peter Grehan for his feedback about RSS APIs
and thanks to Jack Vogel and Navdeep Parhar for the NIC-facing side of the
APIs. These fed into the simple userland API I wrote up here.

Reviewed by: gallatin

Add a WITHOUT_DIALOG src.conf(5) knob

It also turns off dependencies (bsdinstall, bsdconfig, dpv, tzsetup).

Reviewed by: dteske
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7969

Renaming libifc to libifconfig in response to feedback on initial commit of
this library. Sticking to 'libifconfig' (and 'ifconfig_' as function prefix)
should reduce chances of namespace collisions, make it more clear what the
library does, and be more in line with existing libraries.

Submitted by: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Differential Revision: https://reviews.freebsd.org/D7742
Reviewed by: cem, kp

Add libifc, a library implementing core functionality that exists in ifconfig(8) today.

libifc (pronounced lib-ifconfig) aims to be a light abstraction layer between
programs and the kernel APIs for managing the network configuration.
This should hopefully make programs easier to maintain, and reduce code
duplication.

Work will begin on making ifconfig(8) use this library in the near future.

This code is still evolving. The interface should not be considered stable until
it is announced as such.

Submitted By: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D7529

Don't build libdevdctl if MK_CXX is no

Rename ORDERED to BOOTSTRAP since no order is respected in the list.

The directories in SUBDIR_ORDERED are built in parallel, so the name is
misleading.

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Update ^/vendor/NetBSD/tests/dist to a more recent snapshot

Pulled on "Thu Aug 11 18:01:19 PDT 2016"

Vendor import of lldb release_39 branch r276489:
https://llvm.org/svn/llvm-project/lldb/branches/release_39@276489

Vendor import of clang release_39 branch r276489:
https://llvm.org/svn/llvm-project/cfe/branches/release_39@276489

Vendor import of llvm release_39 branch r276489:
https://llvm.org/svn/llvm-project/llvm/branches/release_39@276489

Separate BLACKLIST vs BLACKLIST_SUPPORT properly

Sponsored by: The FreeBSD Foundation

Remove libstdc++ again.

This was mis-merged in r298107 which missed r289389.

Add basic blacklist build support

Reviewed by: rpaulo
Approved by: rpaulo
Relnotes: YES
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5913

Reviewed by: rpaulo
Approved by: rpaulo
Obtained from: NetBSD external/bsd/blacklist @ 20160409
Relnotes: YES
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5912

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

Use the newly minted Makefile.libcompat to implement libsoft libraries
for the armv6 ABI switch. This also make WITH_LIBSOFT functional on
the arm platform. As a transition thing, this seems to work even
without switching the ABI (we basically build the same libraries
twice when MK_LIBSOFT=yes until the ABI cut over next
month). MK_LIBSOFT remains default no.

Revert r296395.

This mistakenly removed the SUBDIR_PARALLEL but even worse is that the install
(and build) order is not correct due to the lack of SUBDIR_DEPEND on the
most critical libraries. The only reason they build correctly now is because
buildworld's 'make libraries' orders them properly.

Pointyhat to: bdrewery
Sponsored by: EMC / Isilon Storage Division

Parallel installation has been safe here since r267511 added SUBDIR_DEPEND.

Sponsored by: EMC / Isilon Storage Division

Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277

Add libpe for elfcopy(1) PE/COFF support

Sponsored by: The FreeBSD Foundation

Remove duplicate line, likely result of a mismerge.

Sponsored by: The FreeBSD Foundation

Style.

Do build libproc and librtld_db for RISC-V as well.

Vendor import of lldb trunk r256945:
https://llvm.org/svn/llvm-project/lldb/trunk@256945

Vendor import of lld trunk r256633:
https://llvm.org/svn/llvm-project/lld/trunk@256633

Make the building of libsmb and mount_smbfs unconditional, now that r292552
has eliminated alignment and endian problems that were making it fail on
some platforms.

PR: 180438
PR: 189415

Start on a new library (libsysdecode) that provides routines for decoding
system call information such as system call arguments. Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.

This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it. One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events. This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events. Instead, this function only generates a string
description for known malloc() and RTLD records.

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D4537

Move obscure lib/ installation of /usr/lib/include symlink to include/.

This avoids the need for an afterinstall: hook and a check for LIBRARIES_ONLY.
It also now respects INCLUDEDIR.

This came in r249484.

Sponsored by: EMC / Isilon Storage Division

Add lib80211, a small (but hopefully soon to grow) set of library
routines to interface with net80211.

This is all from the ifconfig program; the duplicate code from ifconfig
will be removed when it starts using this API.

Differential Revision: https://reviews.freebsd.org/D4290

Always build libevent, which fixes the build for usr.sbin/ypldap after r291230.

The dependency for ypldap is only with WITH_NIS, but libevent is small
enough to not warrant the extra logic and maintenance cost here so just
always build it.

Sponsored by: EMC / Isilon Storage Division

Rename libohash to libopenbsd.

libopenbsd is an internal library which
to bring in compatibility stuff from OpenBSD.
This will allow us to bring in more
OpenBSD utilities into the FreeBSD base system.

We similarly use libnetbsd for bringing in stuff from NetBSD.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D4078

libstdc++ also snook in incorrectly in r267511, despite not being a real
subdir.

Sponsored by: EMC / Isilon Storage Division

Remove unneeded libg++ reference that came in with r267511 based on a removed
comment.

Sponsored by: EMC / Isilon Storage Division

Fix another ++= parsed as '+=', missed in r289384.

Sponsored by: EMC / Isilon Storage Division

Let the SUBDIR_DEPEND*c++ variables actually work rather than being parsed
as a +=. These were safe due to a .WAIT very early on.

Sponsored by: EMC / Isilon Storage Division

Roll WITHOUT_ELFTOOLCHAIN_TOOLS into WITHOUT_TOOLCHAIN

The option was added only to ease the transition from GNU Binutils to
ELF Tool Chain tools, and that process is now complete (for the viable
replacements). Noting the removal in UPDATING is sufficient as we have
not shipped a release with the option.

Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3240

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738

Vendor import of llvm trunk r240225:
https://llvm.org/svn/llvm-project/llvm/trunk@240225

Vendor import of llvm trunk r238337:
https://llvm.org/svn/llvm-project/llvm/trunk@238337

Remove SUBDIR_DEPEND for non-existent libreadline directory

libreadline lives under gnu/lib

MFC after: 1 week

Drop libmandoc and incorporate it into the main mandoc Makefile

This simplifies maintainance of mandoc(1).
Note that the same direction was taken on OpenBSD

Remove redundant csu subdir logic

The appropriate subdirectories are handled by lib/csu/Makefile. There's
no need to duplicate this logic in Makefile.inc1 and lib/Makefile.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2523

Vendor import of lld trunk r233088:
https://llvm.org/svn/llvm-project/lld/trunk@233088

Significant upgrades to sa(4) and mt(1).

The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.

o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.

o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.

This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.

For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.

o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.

o New, extensible tape driver parameter get/set interface.

o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.

o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.

o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.

o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.

o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.

o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.

This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.

rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.

src/share/man/man4/sa.4
Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.

Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.

Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.

Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.

Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.

Add a new MTIOCRBLIM ioctl to get block limits information.

Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().

scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.

Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.

For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.

Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.

Add a partition number to the sa(4) softc.

Improve device departure handling. The previous implementation
led to hangs when the device was open.

If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.

So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.

Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.

Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().

Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.

Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.

This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.

However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.

The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.

This includes definitions for the following new ioctls:

MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */

usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.

Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.

The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.

The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.

Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.

Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.

Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.

Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.

Sponsored by: Spectra Logic
MFC after: 1 month

Initial version of DTrace on ARM32.

Submitted by: Howard Su based on work by Oleksandr Tymoshenko
Reviewed by: ian, andrew, rpaulo, markj

Add a new device control utility for new-bus devices called devctl. This
allows the user to request administrative changes to individual devices
such as attach or detaching drivers or disabling and re-enabling devices.
- Add a new /dev/devctl2 character device which uses ioctls for device
requests. The ioctls use a common 'struct devreq' which is somewhat
similar to 'struct ifreq'.
- The ioctls identify the device to operate on via a string. This
string can either by the device's name, or it can be a bus-specific
address. (For unattached devices, a bus address is the only way to
locate a device.) Bus drivers register an eventhandler to claim
unrecognized device names that the driver recognizes as a valid address.
Two buses currently support addresses: ACPI recognizes any device
in the ACPI namespace via its full path starting with "\" and
the PCI bus driver recognizes an address specification of
'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector
strings supported by pciconf).
- To make it easier to cut and paste, change the PnP location string
in the PCI bus driver to output a full PCI selector string rather
than 'slot=<slot> function=<func>'.
- Add a devctl(3) interface in libdevctl which provides a wrapper around
the ioctls and is the preferred interface for other userland code.
- Add a devctl(8) program which is a simple wrapper around the requests
supported by devctl(3).
- Add a device_is_suspended() function to check DF_SUSPENDED.
- Add a resource_unset_value() function that can be used to remove a
hint from the kernel environment. This is used to clear a
hint.<driver>.<unit>.disabled hint when re-enabling a boot-time
disabled device.

Reviewed by: imp (parts)
Requested by: imp (changing PCI location string)
Relnotes: yes

Add MK_FILE to control whether or not to build file(1), libmagic(3), etc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Conditionalize building radius support into libpam, ppp, etc via
MK_RADIUS_SUPPORT

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add MK_BHYVE knob for building and installing bhyve(4), et al

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Build lib/libgpio if MK_GPIO != no

Fill in corresponding entries for MK_GPIO == no in OptionalObsoleteFiles.inc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Connect libclang_rt to the build, for specific architectures. This
contains the libraries for Address Sanitizer (asan), Undefined Behavior
Sanitizer (ubsan) and Profile Guided Optimization.

ASan is a fast memory error detector. It can detect the following types
of bugs:

Out-of-bounds accesses to heap, stack and globals
Use-after-free
Use-after-return (to some extent)
Double-free, invalid free
Memory leaks (experimental)

Typical slowdown introduced by AddressSanitizer is 2x.

UBSan is a fast and compatible undefined behavior checker. It enables a
number of undefined behavior checks that have small runtime cost and no
impact on address space layout or ABI.

PLEASE NOTE: the sanitizers still have some rough edges on FreeBSD,
particularly on i386. These will hopefully be smoothed out in the
coming time.

Differential Revision: https://reviews.freebsd.org/D1505

Deorbit the IEEE-488/GPIB support.

Move MK_BHYVE/MK_HYPERV to architecture specific sections in Makefiles
instead of introducing an architecture specific section in src.opts.mk

Make it clear in the documentation that the options only affect certain
architectures to reduce user confusion if the knobs are tweaked

A better method for abstracting this out will be done in the future in a
different commit

Discussed with/Requested by: imp

Add knob, MK_MAC_SUPPORT for building items with mac(9) support, like
libugidfw, ugidfw, etc

Build infrastructure for elftoolchain tools

Set WITH_ELFTOOLCHAIN_TOOLS in src.conf to use the elftoolchain version
of the following tools:

* addr2line
* elfcopy (strip / mcs)
* nm
* size
* strings

Reviewed by: bapt (earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1224

Honor MK_AUDIT, MK_FILE, MK_GPIO, and MK_SQLITE3

lib/libgpio is a standalone library, but it seems logical to conditionalize
it with MK_GPIO

Fix typos: MK_RADIUS -> MK_RADIUS_SUPPORT

Implement MK_RADIUS_SUPPORT to make libradius and assorted support optional

Don't build/install lib/libvmmapi if MK_BHYVE == no

Suggested by: grehan

Import libgpio.

This is a thin wrapper around the kernel interface which should make
it easier to write GPIO applications. gpioctl(8) will be converted to
use this library in a separate commit.

Differential Revision: https://reviews.freebsd.org/D1183
Reviewed by: adrian, loos
Discussed on: arm@, embedded@
Relnotes: yes

Vendor import of clang RELEASE_350/final tag r216957 (effectively, 3.5.0 release):
https://llvm.org/svn/llvm-project/cfe/tags/RELEASE_350/final@216957

Vendor import of llvm RELEASE_350/final tag r216957 (effectively, 3.5.0 release):
https://llvm.org/svn/llvm-project/llvm/tags/RELEASE_350/final@216957

Promote SQLite3 as a privatelib as it will also be used by mandoc

While here ensure sqlite3 is using pread(2) and enable the suppot for FTS4

Re-enable dpv(1,3): Introduced via r274116; temporarily disabled
shortly thereafter via r274124 until I could get the right recipe
down w/respect to SUBDIR_DEPEND.

Thanks to: ngie, ian
Reviewed by: ian
MFC after: 21 days
X-MFC-to: stable/10 stable/9
X-MFC-with: 274116 274120 274121 274123 274144 274146

Upon second-thought (following r274144), remove spurious (unused)
line-noise (libdialog never lived in lib/ -- but rather the noise
came from translating a comment that was introduced 16 years ago
via r40306; translation from comment to code occurred via r267511).

MFC after: 3 days
Reviewed by: ngie
X-MFC-to: stable/10

Fix code-typo; introduced by r267511

MFC after: 3 days
X-MFC-to: stable/10

Temporarily _disable_ compilation of dpv(3) and dpv(1).
Will revisit this to find out how to solve the ordering
issue in buildworld (potentially `make -j' specific).

Reviewed by: shurd

Add new libraries/utilities for data throughput visualization.
dpv(3): dialog progress view library
dpv(1): stream data from stdin or multiple paths with dialog progress view
figpar(3): configuration file parsing library

Reviews: D714
Reviewed by: jelischer, shurd
Discussed at: MeetBSD California 2014 Vendor/Dev Summit
Discussed on: -current
MFC after: 21 days
X-MFC-to: stable/10 stable/9

Hook libxo to the build.

Sponsored by: Juniper Networks, Inc.

My previous commit exposed an issue as it fixed a different
issue. lib/atf isn't a prereq_lib, since it isn't required for other
libraries to build. Remove it. The old kludge of always building it
had effectively been retired. Since we don't want to build the
libraries with the tests when we're bootstrapping, invent
MK_TESTS_SUPPORT which normally defaults to the current MK_TESTS
value, except when explicitly defined. Make lib/atf depend on it being
yes. When building the libraries set MK_TESTS to no, and
MK_TESTS_SUPPORT to the current value of MK_TESTS so that later stages
of the build work correctly. This should fix (and does for me)
people's issues with parallel builds racing between lib/atf and
libexec/atf. Since lib/atf is built during the libraries phase, the
race disappears.

Promote libevent to lib/ level and fold ftp-proxy into its parent Makefile.
This allows us to use libevent for other application in the future.

For now libevent is still INTERNALLIB and no shared library is installed.

MFC after: 1 month

Check in first src/tests snapshot from NetBSD anoncvs

Sources were obtained like so:

% export CVSROOT="anoncvs@anoncvs.NetBSD.org:/cvsroot"
% cvs -z9 co -D "09/30/2014 20:45" -P src/tests
% mv src/tests/* tests/dist/.

'*CVS*' has been added to svn:ignore to ease updating periodically from
upstream

Some line ending issues had to be resolved with test outputs and scripts
via dos2unix and by deleting the eol-style property set in usr.bin/sort

Discussed with: rpaulo
Sponsored by: EMC / Isilon Storage Division

Make libohash a proper internallib instead of hidding it in m4

mandoc development seems to be also using ohash so prepare to share the code when
needed

Remove ia64.

This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation

This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h

Discussed at: BSDcan

Create a mechanism for providing fine-grained build order dependencies
during SUBDIR_PARALLEL builds. This augments the coarse .WAIT mechanism,
which is still useful if you've got a situation such as "almost everything
depends on A and B".

Because the parallel subdir mechanism uses non-obvious mangling of
target names, which should probably remain a private detail of the
implementation, it's not easy to do things like "libfoo: libbar", so
instead the new mechanism lets you set a variable that lists dependencies:

SUBDIR_DEPEND_libfoo= libgroodah libpouet

Note that while I'm using libraries as an example here, it really has
nothing to do with the generated library files. This is really saying
"build in directory libfoo after building in the libgroodah and libpouet
directories."

This updates lib/Makefile with dependency information based on the old
almost-accurate comment block and by combing through lib/* makefiles
looking for LDADD dependencies to other libraries within lib/*.

Reviewed by: Jia-Shiun Li <jiashiun@gmail.com>

Attach the CUSE library and kernel module to the default FreeBSD
builds. Bump the FreeBSD version number.

Sprinkle a few more .WAITs into the mix after csu, libc, msun and the
early built libraries. This should be sufficient for most cases and
has eliminated the issues I've seen with high -j builds. Races likely
still remain, but this knocks the problem down a notch.

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

lib: Remove duplicate SUBDIR libnv.

With SUBDIR_PARALLEL, duplicates in ${SUBDIR} cause harmless but verbose
warnings.

Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
all the SUBDIR entries in parallel, instead of serially. Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.

This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.

I tested this on a 24-core machine, with make -j48 buildworld (N = 6):

before stddev after stddev
======= ====== ======= ======
real time 1741.1 16.5 959.8 2.7
user time 12468.7 16.4 14393.0 16.8
sys time 1825.0 54.8 2110.6 22.8

(user+sys)/real 8.2 17.1

E.g. the build was approximately 45% faster in real time. On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive. But at least you can now almost max out a machine with
buildworld!

Submitted by: jilles
MFC after: 2 weeks

Remove IPX support.

IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.

Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.

Add 'libstatfoo' from Sam Leffler.

Remove libyaml, it has been replaced by libucl

Branch libucl to the build

Vendor import of clang release_34 branch r197841 (effectively, 3.4 RC3):
https://llvm.org/svn/llvm-project/cfe/branches/release_34@197841

Vendor import of llvm release_34 branch r197841 (effectively, 3.4 RC3):
https://llvm.org/svn/llvm-project/llvm/branches/release_34@197841

Please welcome casperd daemon. It (and its services) will be responsible for
giving access to functionality that is not available in capability mode
sandbox. The functionality can be precisely restricted.

Start with the following services:
- system.dns - provides API compatible to:
- gethostbyname(3),
- gethostbyname2(3),
- gethostbyaddr(3),
- getaddrinfo(3),
- getnameinfo(3),
- system.grp - provides getgrent(3)-compatible API,
- system.pwd - provides getpwent(3)-compatible API,
- system.random - allows to obtain entropy from /dev/random,
- system.sysctl - provides sysctlbyname(3-compatible API.

Sponsored by: The FreeBSD Foundation

Move my simple logging API to a separate library. It is now already used
by hastctl(8), hastd(8) and auditdistd(8) and will soon be also used
by casperd(8) and its services. There is no documentation and pjdlog.h
header file is not installed in /usr/include/ to keep it private.
Unfortunately we don't have /lib/private/ at this point, only
/usr/lib/private/, so the library is installed in /lib/.

Sponsored by: The FreeBSD Foundation

Attempt to move the POSIX iconv* symbols out of runtime linker space.
FreeBSD systems usually implemented this as a third party module and
our implementation hasn't played as nicely with the old way as it could
have.

To that end:
* Rename the iconv* symbols in libc.so.7 to have a __bsd_ prefix.
* Provide .symver compatability with existing 10.x+ binaries that
referenced the iconv symbols. All existing binaries should work.
* Like on Linux/glibc systems, add a libc_nonshared.a to the ldscript
at /usr/lib/libc.so.
* Move the "iconv*" wrapper symbols to libc_nonshared.a

This should solve the runtime ambiguity about which symbols resolve
to where. If you compile against the iconv in libc, your runtime
dependencies will be unambiguous.

Old 9.x libraries and binaries will always resolve against their
libiconv.so.3 like they did on 9.x. They won't resolve against libc.

Old 10.x binaries will be satisified by the .symver helpers.

This should allow ports to selectively compile against the libiconv
port if needed and it should behave without ambiguity now.

Discussed with: kib

Bring in libnv library for managing name/value pairs. The following types
are currently supported:

- NV_TYPE_NULL - only name, no data;
- NV_TYPE_BOOL - boolean (true or false);
- NV_TYPE_NUMBER - 64bit unsigned integer;
- NV_TYPE_STRING - C string;
- NV_TYPE_NVLIST - nested nvlist;
- NV_TYPE_DESCRIPTOR - file descriptor;
- NV_TYPE_BINARY - binary data.

For detailed documentation and examples see nv(3) manual page.

Sponsored by: The FreeBSD Foundation

Install category Kyuafiles from their category directories.

Move the installation of /usr/tests/lib/Kyuafile from src/tests/lib/
to src/lib/. This is to keep the src/tests/ hierarchy unaware of the
rest of the tree, which makes things clearer in general. In particular:

1) Everything related to the construction of /usr/tests/lib/ is kept
in src/lib/. There is no need to think about different directories
and how they relate to each other. (The same applies for libexec,
usr.bin, etc. but these are not yet handled.)

2) src/tests becomes the place to keep cross-functional test programs
and nothing else, which also helps in simplifying things.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

Subsume the functionality of MK_ATF into MK_TESTS.

There is no reason to keep the two knobs separate: if tests are
enabled, the ATF libraries are required; and if tests are disabled,
the ATF libraries are not necessary. Keeping the two just serves
to complicate the build.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

Remove the WITH_LIBICONV_COMPAT hack that seems to do more harm than
good. This caused libc to spoof the ports libiconv namespace and
provide a colliding libiconv.so.3 to fool rtld. This should have
been removed some time ago.

Extract devctl event processing support out of zfsd and into a new library:
libdevctl.

etc/mtree/BSD.include.dist:
Add the /usr/include/devctl directory to the base system.

lib/Makefile:
lib/libdevctl/Makefile:
Build support for the new library.

lib/libdevctl/consumer.cc:
lib/libdevctl/consumer.h:
cddl/sbin/zfsd/zfsd.cc:
cddl/sbin/zfsd/zfsd.h:
New class, DevCtl::Consumer, from functionality extracted
from ZfsDaemon, which can connect to devd, parse an event
stream, and invoke event handlers.

lib/libdevctl/event.cc:
lib/libdevctl/event.h:
cddl/sbin/zfsd/dev_ctl_event.cc:
cddl/sbin/zfsd/dev_ctl_event.h:
Strip Zfsd specific event handling from Zfsd's event classes
to create DevCtl::Event, DevCtl::DevfsEvent, and
DevCtl::ZfsEvent.

lib/libdevctl/event_buffer.cc:
lib/libdevctl/event_buffer.h:
cddl/sbin/zfsd/zfsd.cc:
cddl/sbin/zfsd/zfsd.h:
DevCtl::EventBuffer: buffer for incoming devctl stream used
to parse/extract individual event strings.

lib/libdevctl/event_factory.cc:
lib/libdevctl/event_factory.h:
cddl/sbin/zfsd/zfsd.cc:
cddl/sbin/zfsd/zfsd.h:
DevCtl::EventFactory: Method map for converting event
strings to Devctl::Event objects.

lib/libdevctl/exception.cc:
lib/libdevctl/exception.h:
cddl/sbin/zfsd/zfsd_exception.cc:
cddl/sbin/zfsd/zfsd_exception.h:
DevCtl::Exception and DevCtl::ParseException, the exceptions
which are thrown by this library.

lib/libdevctl/guid.cc:
lib/libdevctl/guid.h:
cddl/sbin/zfsd/guid.cc:
cddl/sbin/zfsd/guid.h:
DevCtl::Guid: Helper routines for dealing with 64bit GUIDs
such as found in Zfs pools and vdevs.

lib/libdevctl/reader.cc:
lib/libdevctl/reader.h:
cddl/sbin/zfsd/zfsd.cc:
cddl/sbin/zfsd/zfsd.h:
DevCtl::Reader class hierarchy. Used to direct a Consumer
to an event stream (e.g. from devd or a local/saved file).

cddl/sbin/zfsd/zfsd_event.cc:
cddl/sbin/zfsd/zfsd_event.h:
Zfsd specialization of DevCtl::Event types.

cddl/sbin/zfsd/zfsd_exception.cc:
cddl/sbin/zfsd/zfsd_exception.h:
Zfsd specific exception types, now derived from
DevCtl::Excpetion.

cddl/sbin/zfsd/Makefile:
cddl/sbin/zfsd/callout.cc:
cddl/sbin/zfsd/callout.h:
cddl/sbin/zfsd/case_file.cc:
cddl/sbin/zfsd/case_file.h:
cddl/sbin/zfsd/vdev.cc:
cddl/sbin/zfsd/vdev.h:
cddl/sbin/zfsd/vdev_iterator.cc:
cddl/sbin/zfsd/vdev_iterator.h:
cddl/sbin/zfsd/zfsd.cc:
cddl/sbin/zfsd/zfsd.h:
cddl/sbin/zfsd/zfsd_event.cc:
cddl/sbin/zfsd/zfsd_event.h:
cddl/sbin/zfsd/zfsd_exception.cc:
cddl/sbin/zfsd/zfsd_exception.h:
cddl/sbin/zfsd/zfsd_main.cc:
cddl/sbin/zfsd/zpool_list.cc:
cddl/sbin/zfsd/zpool_list.h:
Miscelaneuous refactoring to support using libdevctl instead
of local implementation.

cddl/sbin/zfsd/Makefile:
cddl/sbin/zfsd/Makefile.common:
tools/regression/zfsd/Makefile:
Export common build information for zfsd and its unit tests
to a "Makefile.common".

Submitted by: gibbs
Approved by: ken (mentor)
Sponsored by: Spectra Logic Corporation

Remove BIND.

Approved by: re (gjb)

Move libldns to the correct (ordered) library list.

Approved by: re (blanket)

Build and install the Unbound caching DNS resolver daemon.

Approved by: re (blanket)

Connect libexecinfo to the build

Sponsored by: DARPA, AFRL

The iconv in libc did two things - implement the standard APIs, the GNU
extensions and also tried to be link time compatible with ports libiconv.
This splits that functionality and enables the parts that shouldn't
interfere with the port by default.

WITH_ICONV (now on by default) - adds iconv.h, iconv_open(3) etc.
WITH_LIBICONV_COMPAT (off by default) adds the libiconv_open etc API, linker
symbols and even a stub libiconv.so.3 that are good enough to be able
to 'pkg delete -f libiconv' on a running system and reasonably expect it
to work.

I have tortured many machines over the last few days to try and reduce
the possibilities of foot-shooting as much as I can. I've successfully
recompiled to enable and disable the libiconv_compat modes, ports that use
libiconv alongside system iconv etc. If you don't enable the
WITH_LIBICONV_COMPAT switch, they don't share symbol space.

This is an extension of behavior on other system. iconv(3) is a standard
libc interface and libiconv port expects to be able to run alongside it on
systems that have it.

Bumped osreldate.

Import IP-Filter 5.1.2 into vendor branches using the existing license that
the current version of IP-Filter in FreeBSD is under as per email received
from Darren Reed on Mon, 08 Jul 2013 23:54:16 +1000.

Approved by: glebius (Mentor), Darren Reed <darrenr@reed.wattle.id.au>

As per the developers handbook (5.3.1 step 1), prepare the vendor trees for
import of new ipfilter vendor sources by flattening them.

To keep the tags consistent with dist, the tags are also flattened.

Approved by: glebius (Mentor)

- Trim an unused and bogus Makefile for mount_smbfs.
- Reconnect with some minor modifications, in particular now selsocket()
internals are adapted to use sbintime units after recent'ish calloutng
switch.

Wrap the creation of the ../include link in /usr/lib in
!defined(LIBRARIES_ONLY) so it is only created once on architectures
with 32-bit compat support.

Replace ln -fhs with ${INSTALL_SYMLINK} to the link is logged in the
METALOG.

An even more refined version of r249484, until we can come up with
a good fix for the -print-file-name=include breakage.

As suggested by Andrey Chernov.

Fix the symlink creation from r249484 so that repeated
installs work correctly.

Suggested by Tijl Coosemans.

Install a symlink
/usr/lib/include ==> /usr/include

This fixes -print-file-name=include in clang (and is
arguably a better way to fix the same issue in GCC than
the change I made in r231336).

MFC after: 1 week

Vendor import of clang trunk r178860:
http://llvm.org/svn/llvm-project/cfe/trunk@178860

Vendor import of llvm trunk r178860:
http://llvm.org/svn/llvm-project/llvm/trunk@178860

Sort by MK_* knob like the comment says

MFC after: 1 week

With the old sade removed, libdisk is no longer used by anything in HEAD
and uses a number of problematic pre-gpart interfaces. Since it has been
entirely obsoleted by interfaces in geom, remove it.

Add libnetbsd, a thin compatibility layer intended to allow a limited
set of NetBSD software to compile as part of the FreeBSD build with
little or no modifiction. It is built as a static library and not
installed for general use. Likewise, its header files are not
installed.

Sponsored by: DARPA, AFRL

Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.

There is one known issue: Some probes will display an error message along the
lines of: "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit. I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded. Volunteers are welcome.

MFC after: 1 month

Add ATF to the build. This is may be a bit rought around the egdes,
but committing it helps to get everyone on the same page and makes
sure we make progress.

Tinderbox breakages that are the result of this commit are entirely
the committer's fault -- in other words: buildworld testing on amd64
only.

Credits follow:

Submitted by: Garrett Cooper <yanegomi@gmail.com>
Sponsored by: Isilon Systems
Based on work by: keramida@
Thanks to: gnn@, mdf@, mlaier@, sjg@
Special thanks to: keramida@

Apply local patches to mandoc and connect it to the build.

- adds a couple more library strings used in the tree
- changes some more to the current groff spelling
- changes page footer to match groff style

Disconnect non-MPSAFE SMBFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netsmb, which is a base
requirement for SMBFS.

In the while SMBFS regular users can use FUSE interface and smbnetfs
port to work with their SMBFS partitions.

Also, there are ongoing efforts by vendor to support in-kernel smbfs,
so there are good chances that it will get relinked once properly locked.

This is not targeted for MFC.

Disconnect non-MPSAFE NWFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netncp, which is a base
requirement for NWFS.

In the possibility of a future maintenance of the code and later
readd to the FreeBSD base, maybe we should think about a better location
for netncp. I'm not entirely sure the / top location is actually right,
however I will let network people to comment on that more specifically.

This is not targeted for MFC.

Vendor import of clang trunk r161861:
http://llvm.org/svn/llvm-project/cfe/trunk@161861

Import work done under project/nand (@235533) into head.

The NAND Flash environment consists of several distinct components:
- NAND framework (drivers harness for NAND controllers and NAND chips)
- NAND simulator (NANDsim)
- NAND file system (NAND FS)
- Companion tools and utilities
- Documentation (manual pages)

This work is still experimental. Please use with caution.

Obtained from: Semihalf
Supported by: FreeBSD Foundation, Juniper Networks

Vendor import of clang release_31 branch r155985:
http://llvm.org/svn/llvm-project/cfe/branches/release_31@155985

Import stdbuf(1) and the shared library it relies on.
This tool changes the default buffering behaviour of standard
stdio streams.

It only works on dynamic binaries. To make it work for static
ones it would require cluttering stdio because there no single
entry point.

PR: 166660
Reviewed by: current@, jhb
Approved by: kib (mentor)
MFC after: 1 week

Vendor import of clang trunk r154661:
http://llvm.org/svn/llvm-project/cfe/trunk@r154661

Build libproc and librtld_db for MIPS

- Do not build libcom_err and compile_et when kerberos is disabled. They
depends on several heimdal libraries and not used by anything but kerberos
tools.

Fix typo in Makefile comment.

MFC after: 3 days

Add libstdthreads.

This library implements the C11 threads interface on top of the pthreads
library. As discussed on the lists, the preferred way to implement
this, is as a separate library.

It is unlikely that these functions will be used a lot in the future. It
would have been easier if the C11 working group standardized (a subset
of) pthreads and clock_nanosleep(). Having it as a separate library
allows the embedded people to omit it from their system.

Discussed on: arch@, threads@

Fix breakage after r227983; lib/libcxxrt still got built, because it was
not disabled in the usual way (by adding it to __DEFAULT_NO_OPTIONS in
share/mk/bsd.own.mk), and because the test for MK_LIBCPLUSPLUS in
Makefile.inc1 was incorrect.

Pointy hat to: dim

Vendor import of llvm release_30 branch r142614:
http://llvm.org/svn/llvm-project/llvm/branches/release_30@142614

Now that its only consumer is gone (sysinstall), remove libftpio as well.

Remove a redundant check for libncp.

Submitted by: Alexander Sack (asack at niksun dot com)

Vendor import of clang trunk r135360:
http://llvm.org/svn/llvm-project/cfe/trunk@135360

Vendor import of clang trunk r132879:
http://llvm.org/svn/llvm-project/cfe/trunk@132879

Backout libinstall.a -> libpkg commit.

Discussed with: erwin, brooks, bapt

Import of bhyve hypervisor and utilities, part 1.
vmm.ko - kernel module for VT-x, VT-d and hypervisor control
bhyve - user-space sequencer and i/o emulation
vmmctl - dump of hypervisor register state
libvmm - front-end to vmm.ko chardev interface

bhyve was designed and implemented by Neel Natu.

Thanks to the following folk from NetApp who helped to make this available:
Joe CaraDonna
Peter Snyder
Jeff Heller
Sandeep Mann
Steve Miller
Brian Pawlowski

- Commit work from libprocstat project. These patches add support for runtime
file and processes information retrieval from the running kernel via sysctl
in the form of new library, libprocstat. The library also supports KVM backend
for analyzing memory crash dumps. Both procstat(1) and fstat(1) utilities have
been modified to take advantage of the library (as the bonus point the fstat(1)
utility no longer need superuser privileges to operate), and the procstat(1)
utility is now able to display information from memory dumps as well.

The newly introduced fuser(1) utility also uses this library and able to operate
via sysctl and kvm backends.

The library is by no means complete (e.g. KVM backend is missing vnode name
resolution routines, and there're no manpages for the library itself) so I
plan to improve it further. I'm commiting it so it will get wider exposure
and review.

We won't be able to MFC this work as it relies on changes in HEAD, which
was introduced some time ago, that break kernel ABI. OTOH we may be able
to merge the library with KVM backend if we really need it there.

Discussed with: rwatson

Vendor import of clang trunk r130700:
http://llvm.org/svn/llvm-project/cfe/trunk@130700

Add the BSD-licensed Citrus iconv to the base system with default off
setting. It can be built by setting the WITH_ICONV knob. While this
knob is unset, the library part, the binaries, the header file and
the metadata files will not be built or installed so it makes no impact
on the system if left turned off.

This work is based on the iconv implementation in NetBSD but a great
number of improvements and feature additions have been included:

- Some utilities have been added. There is a conversion table generator,
which can compare conversion tables to reference data generated by
GNU libiconv. This helps ensuring conversion compatibility.
- UTF-16 surrogate support and some endianness issues have been fixed.
- The rather chaotic Makefiles to build metadata have been refactored
and cleaned up, now it is easy to read and it is also easier to add
support for new encodings.
- A bunch of new encodings and encoding aliases have been added.
- Support for 1->2, 1->3 and 1->4 mappings, which is needed for
transliterating with flying accents as GNU does, like "u.
- Lots of warnings have been fixed, the major part of the code is
now WARNS=6 clean.
- New section 1 and section 5 manual pages have been added.
- Some GNU-specific calls have been implemented:
iconvlist(), iconvctl(), iconv_canonicalize(), iconv_open_into()
- Support for GNU's //IGNORE suffix has been added.
- The "-" argument for stdin is now recognized in iconv(1) as per POSIX.
- The Big5 conversion module has been fixed.
- The iconv.h header files is supposed to be compatible with the
GNU version, i.e. sources should build with base iconv.h and
GNU libiconv. It also includes a macro magic to deal with the
char ** and const char ** incompatibility.
- GNU compatibility: "" or "char" means the current local
encoding in use
- Various cleanups and style(9) fixes.

Approved by: delphij (mentor)
Obtained from: The NetBSD Project
Sponsored by: Google Summer of Code 2009

Vendor import of clang trunk r126079:
http://llvm.org/svn/llvm-project/cfe/trunk@126079

Vendor import of llvm trunk r126079:
http://llvm.org/svn/llvm-project/llvm/trunk@126079

Replace libgcc.a by libcompiler_rt.a.

libcompiler_rt.a is a BSD licensed C language runtime, which implements
many routines which are linked into binaries on architectures where
certain functionality is missing (e.g. 64 bits mul/div on i386).

Unfortunately, libcompiler_rt cannot replace libgcc entirely. Certain
features, such as an unwinder for exception handling, are missing.
That's why only libgcc.a is replaced for now, because this one does seem
to be complete.

Tested by: rene (amd64), nwhitehorn (powerpc), droso (i386 exprun)
and many others. Thanks!
Obtained from: user/ed/compiler-rt

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Vendor import of clang r114020 (from the release_28 branch):
http://llvm.org/svn/llvm-project/cfe/branches/release_28@114020

Approved by: rpaulo (mentor)

Repair some build breakage introduced in r211725 and garbage collect some
code made obsolete in the same commit.

Change the logic here to match Makefile.inc1. Having it in two places
suggests an opportunity for refactoring :)

Submitted by: nathanw@

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

Build librtld_db.

Sponsored by: The FreeBSD Foundation

Simplify the Makefile. The i386 and amd64 sections are equal.

Sponsored by: The FreeBSD Foundation

libproc is going to be i386/amd64 only with the next update.

Sponsored by: The FreeBSD Foundation

Update clang to r108243.

Build libsmb should be on powerpc64 as well.

Don't build Clang libs during lib32 build.

This should massively reduce the buildworld time on amd64.

Pointy hat to: me

Hook clang into the build on i386/amd64/powerpc.

Approved by: ed (mentor)

MFC r207842, r207844, r208099:

MFC r207842:
Import of liblzma, xz, xzdec, lzmainfo from vendor branch
Add support for xz and lzma to lesspipe.sh (xzless, lzless)

MFC r207844:
Add two public headers missing in r207842
Adjust CFLAGS for lzmainfo, xz, xzdec

MFC r208099:
Add versioned symbols to liblzma
Use default SHLIB_MAJOR.

Approved by: delphij (mentor)

MFC r204738, r205113 (imp):

MFC r204738:
Remove stale references to libkrb5.
Rejigger the SUBDIR setting a smidge: we now set all the libraries that depend
on something else, and then SUBDIR+= the rest.

MFC r205113:
Make this conform to the other top-level Makefile subdir listings with
one file per line.

Approved by: delphij (mentor)

- Take libinstall.a out of pkg_install and make it a proper shared library.
- Rework the wrapper support to check libpkg version as well as pkg_install
version.
- Add libfetch to _prebuild_libs.
- There are no new features introduced.

Notes: the API is not stable, so basically, do not use libpkg in your
projects for now. Also there's no manpage for libpkg yet, because the API
will change drastically. I repeat, do not use libpkg for now.

Make this conform to the other top-level Makefile subdir listings with
one file per line.

Suggested by: bde

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Remove stale references to libkrb5.
Rejigger the SUBDIR setting a smidge: we now set all the libraries that depend
on something else, and then SUBDIR+= the rest.

A separate commit will fix the SUBDIR style to be the same as the rest
of the tree.

MFC x86emu/x86bios emulator and make previously i386 only dpms and vesa
framebuffer driver, etc. work on FreeBSD/amd64.

A significant amount of improvements were done by jkim@ during the recent
months to make vesa(4) work better, over the initial code import. This
work is based on OpenBSD's x86emu implementation and contributed by
paradox <ddkprog yahoo com> and swell.k at gmail com.

Hopefully I have stolen all their work to 8-STABLE :)

All bugs in this commit are mine, as usual.

Update clang to r96341.

Introduce libefi -- a library around EFI runtime services and protocols.
This first commit brings 3 functions for enumerating, retrieving,
adding, removing and modifying EFI variables. The immediate use of these
include the insertion of a new boot option as part of the installation
process.

This library uses ioctl(2) requests implemented by io(4) to pass the
requests down through the kernel to EFI. These ioctl requests are only
implemented on ia64, so libefi is currently only enabled on ia64. The
interface is generic and io(4) on mad64/i386 can easily be taught to
handle these once EFI support has been added to the kernel there.

Connect smbfs related libraries and tool on sparc64.

Reviewed by: marius

Update clang to r94309.

Remove libulog from the bootstrap again.

libulog now only provides functions that are used by various packages
from the ports tree, namely the libutempter ones. There is no reason to
link it into the crunch/fixit binaries anymore.

Convert pam_lastlog(8) to libulog.

The information used by the "Last login:"-line is obtained by using
ulog_setutxfile(3) to switch to the lastlog database. Login and logout
are performed using the utility functions ulog_login(3) and
ulog_logout(3).

This also means we must build libulog during bootstrap.

Approved by: des

Add a new library: libulog.

One of the things I really want to do, is to get rid of the limitations
of our current utmp(5) mechanism:

- It only allows 8 byte TTY device names.
- The hostname only allows 16 bytes of storage.

I'm not a big fan of <utmpx.h>, but I think we should at least try to
add parts of it. Unfortunately we cannot implement <utmpx.h>, because we
miss various fields, such as ut_id, ut_pid, etc. The API provided by
libulog shares some similarities with <utmpx.h>, so it shouldn't be too
hard to port these applications eventually. In most simple cases, it
should just be a matter of removing the ulog_ prefix everywhere.

As a bonus, it also implements a function called ulog_login_pseudo(),
which allows unprivileged applications to write log entries, provided
they have a valid file descriptor to a pseudo-terminal master device.

libulog will allow a smoother transition to a new file format by adding
a library interface to deal with utmp/wtmp/lastlog files. I initially
thought about adding the functionality to libutil, but because I'm not
planning on keeping this library around forever, we'd better keep it
separated.

Next items on the todo list:

1. Port applications in the base system (and ports) to libulog, instead
of letting them use <utmp.h>.
2. Remove <utmp.h>, implement <utmpx.h> and reimplement this library on
top.
3. Port as many applications as possible back to <utmpx.h>.

Update clang to r84119.

Update llvm to r84119.

- Teach vesa(4) and dpms(4) about x86emu. [1]
- Add vesa kernel options for amd64.
- Connect libvgl library and splash kernel modules to amd64 build.
- Connect manual page dpms(4) to amd64 build.
- Remove old vesa/dpms files.

Submitted by: paradox <ddkprog yahoo com> [1], swell k at gmail.com
(with some minor tweaks)

Import LLVM 74788.

Import LLVM r74383.

Add libjail, a (somewhat) simpler interface to the jail_set and jail_get
system calls and the security.jail.param sysctls.

Approved by: bz (mentor)

Import Clang, at r72732.

Import LLVM, at r72732.

Commit missed file in r189587, update directory name for libusb.

Spotted by: rdivacky

src/sys/dev/usb2/controller/uss820dci_pccard.c
src/sys/dev/usb2/core/usbdevs
src/sys/dev/usb2/include/urio2_ioctl.h
src/sys/dev/usb2/storage/ustorage2_fs.h

These files are not used any more.

src/usr.sbin/Makefile
src/etc/mtree/BSD.include.dist
src/include/Makefile
src/lib/Makefile
src/share/man/man7/hier.7
src/share/mk/bsd.libnames.mk
src/etc/mtree/BSD.include.dist

Make "usbconfig" and "libusb20" a part of the default build.

src/sys/dev/usb/rio500_usb.h
src/sys/dev/usb2/storage/urio2.c

Use common include file.

src/sys/dev/usb2/bluetooth/ng_ubt2.c

Make USB bluetooth depend on "ng_hci" module.

src/sys/dev/usb2/controller/ehci2.c
src/sys/dev/usb2/controller/ehci2.h

Patches for Marvell EHCI.

src/sys/dev/usb2/core/usb2_busdma.c

Bugfix for 64-bit platforms. Need to unload the previously loaded DMA
map and some cleanup regarding some corner cases.

src/sys/dev/usb2/core/usb2_core.h
src/sys/dev/usb2/core/usb2_dev.c
src/sys/dev/usb2/core/usb2_dev.h

Bugfix for libusb filesystem interface.

New feature: Add support for filtering device data at the expense of the
userland process.

Add some more comments.

Some minor code styling.

Remove unused function, usb2_fifo_get_data_next().

Fix an issue about "fifo_index" being used instead of "ep_index".

src/sys/dev/usb2/core/usb2_device.c
src/sys/dev/usb2/core/usb2_generic.c

Bugfix for Linux USB compat layer. Do not free non-generic FIFOs when
doing an alternate setting.

Cleanup USB IOCTL and USB reference handling.
Fix a corner case where USB-FS was left initialised after
setting a new configuration or alternate setting.

src/sys/dev/usb2/core/usb2_hub.c

Improvement: Check all USB HUB ports by default at least one time.

src/sys/dev/usb2/core/usb2_request.c

Bugfix: Make sure destination ASCII string is properly zero terminated
in all cases.

Improvement: Skip invalid characters instead of replacing with a dot.

src/sys/dev/usb2/core/usb2_util.c
src/sys/dev/usb2/image/uscanner2.c

Spelling.

src/sys/dev/usb2/include/Makefile

Share "usbdevs" with the old USB stack.

src/sys/dev/usb2/include/usb2_devid.h
src/sys/dev/usb2/include/usb2_devtable.h

Regenerate files.

Alfred: Please fix the RCS tag at the top.

src/sys/dev/usb2/include/usb2_ioctl.h

Fix compilation of "kdump".

src/sys/dev/usb2/serial/ubsa2.c
src/sys/dev/usb2/serial/ugensa2.c

Remove device ID's which will end up in a new 3G driver.

src/sys/dev/usb2/sound/uaudio2.c

Correct a debug printout.

src/sys/dev/usb2/storage/umass2.c

Sync with old USB stack.

src/lib/libusb20/libusb20.3

Add more documentation.

src/lib/libusb20/libusb20.c

Various bugfixes and improvements.

src/usr.sbin/usbconfig/dump.c
src/usr.sbin/usbconfig/usbconfig.c

New commands for dumping strings and doing custom USB requests from
the command line.

Remove keyword requirements from generated files:
"head/sys/dev/usb2/include/usb2_devid.h"
"head/sys/dev/usb2/include/usb2_devtable.h"

- Import the HEAD csup code which is the basis for the cvsmode work.

add new build knobs and jigger some existing controls to improve
control over the result of buildworld and installworld; this especially
helps packaging systems such as nanobsd

Reviewed by: various (posted to arch)
MFC after: 1 month

When I flattened the dist tree, some files that had been removed in prior
vendor imports were inadvertantly resurrected. Re-remove them.

Add an implementation of the RPCSEC_GSS authentication protocol for RPC. This
is based on an old implementation from the University of Michigan with lots of
changes and fixes by me and the addition of a Solaris-compatible API.

Sponsored by: Isilon Systems
Reviewed by: alfred

Properly flatten openssh/dist.

Remove netatm from HEAD as it is not MPSAFE and relies on the now removed
NET_NEEDS_GIANT. netatm has been disconnected from the build for ten
months in HEAD/RELENG_7. Specifics:

- netatm include files
- netatm command line management tools
- libatm
- ATM parts in rescue and sysinstall
- sample configuration files and documents
- kernel support as a module or in NOTES
- netgraph wrapper nodes for netatm
- ctags data for netatm.
- netatm-specific device drivers.

MFC after: 3 weeks
Reviewed by: bz
Discussed with: bms, bz, harti

Hook libdwarf and libproc into the build.

- Remove libkse and related support code in libpthread from the build.
Don't remove the files yet. Kernel support will be removed shortly.

- Fix setting of MK_GSSAPI option by bsd.own.mk; its value should
default to the value of MK_KERBEROS unless set explicitly by
WITH_GSSAPI/WITHOUT_GSSAPI. (This introduces another type of
MK_* variables which itself is questionable.)

- Teach tools/build/options/makeman script that generates the
src.conf(5) manpage about the new type of MK_* variables.

- Fix broken logic in lib/Makefile.

Hide the building and installation of libgssapi behind the
WITHOUT_KERBEROS knob. While GSS can be used for other things
some third party software (most notably ports/x11/kdelibs3)
takes the presence of libgssapi as an indication that kerberos
is available, and attempts to link with the kerberos libs. If
they are not available, the build will fail.

Because you might want to use GSS but not kerberos, add a knob
to re-enable it if WITHOUT_KERBEROS is present.

Document the new knob, and the new behavior of WITHOUT_KERBEROS.

Not objected and/or generally agreed to by: freebsd-arch

Problem discussed/analyzed in:
PR: ports/116484

Build libthread_db for arm as well.

MFC after: 1 week

Add a period for yar@.

Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by: ru
Approved by: re(kensmith)

Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by: re(kensmith)

Disconnect netatm from the build as it is not MPSAFE and relies on
NET_NEEDS_GIANT, which will shortly be removed. This is done in a
away that it may be easily reattached to the build before 7.1 if
appropriate locking is added. Specifics:

- Don't install netatm include files
- Disconnect netatm command line management tools
- Don't build libatm
- Don't include ATM parts in rescue or sysinstall
- Don't install sample configuration files and documents
- Don't build kernel support as a module or in NOTES
- Don't build netgraph wrapper nodes for netatm

This removes the last remaining consumer of NET_NEEDS_GIANT.

Reviewed by: harti
Discussed with: bz, bms
Approved by: re (kensmith)

Make sure libc is installed before all other shared libraries. Other
libraries might depend on versioned symbols from libc.

Switch to new ncurses build glue

Approved by: delphij

MFP4: Add an implementation of the ELF(3) and GELF(3) API set.

Bump __FreeBSD_version.

Reviewed by: jb

Removed libc_r build support.

Remove alpha left-overs.

Build libsmb and smbutil on ia64.

Build libthread_db on PowerPC.

Extend coverage of the MK_IPX build option to the following:

- <netipx> headers [1]
- IPX library (libipx)
- IPX support in ifconfig(8)
- IPXrouted(8)
- new MK_NCP option

New MK_NCP build option controls:

- <netncp> and <fs/nwfs> headers
- NCP library (libncp)
- ncplist(1) and ncplogin(1)
- mount_nwfs(8)
- ncp and nwfs kernel modules

User knobs: WITHOUT_IPX, WITHOUT_IPX_SUPPORT, WITHOUT_NCP.

[1] <netsmb/netbios.h> unconditionally uses <netipx> headers
so they are still installed. This needs to be dealt with.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Connect librt to buildworld.

Add libbsm to the library build.

Obtained from: TrustedBSD Project

Add a dummy arm Write_Disk() function, and compile libdisk on arm.

Add a new extensible GSS-API layer which can support GSS-API plugins,
similar the the Solaris implementation. Repackage the krb5 GSS mechanism
as a plugin library for the new implementation. This also includes a
comprehensive set of manpages for the GSS-API functions with text mostly
taken from the RFC.

Reviewed by: Love Hörnquist Åstrand <lha@it.su.se>, ru (build system), des (openssh parts)

Disconnect libc_r from buildworld, it is still kept in the tree to
provide some baseline references, but users are encouraged to use
libpthread or libthr in real world.

Discussed on: arch@

Connect smbfs build on powerpc.

Hook libmemstat(3) up to the build.

Build libpmc on all architectures (FWIW :-)

Ok'd by: jkoshy@

Bring a working snapshot of hwpmc(4), its associated libraries, userland utilities
and documentation into -CURRENT.

Bump FreeBSD_version.

Reviewed by: alc, jhb (kernel changes)

Connect libthr and libthread_db to make buildworld.

Prepare for importing my 1:1 threading work, disconnect libthr and
libthread_db from make buildworld.

Tie libgpib into the build (unless NO_GPIB)

NOCRYPT -> NO_CRYPT

NOATM -> NO_ATM

NOLIBC_R -> NO_LIBC_R
NOLIBPTHREAD -> NO_LIBPTHREAD
NOLIBTHR -> NO_LIBTHR

Remove stub libxpg4. All functionality was merged into libc a long time ago.

Add knob NO_NIS (fka NO_YP_LIBC) and make world compileable when set.
If turned on no NIS support and related programs will be built.

Lost parts rediscovered by: Danny Braniss <danny at cs.huji.ac.il>
PR: bin/68303
No objections: des, gshapiro, nectar
Reviewed by: ru
Approved by: rwatson (mentor)
MFC after: 2 weeks

Help Tinderbox and remove libautofs

Fix build in the !NOATM case by using the begemot library in place of
using libisc which was a part of BIND8.

Discussed with: des, re, dougb
Submitted by: harti (one part)
Reviewed by: harti (previous version)

Switch from BIND 8 to BIND 9.

Submitted by: (in part) dougb@, trhodes@
Reviewed by: dougb@, trhodes@, re@
MFC after: 5 days

Hook libautofs to the build.

Start the de-orbital burn of our previous FreeBSD version compat libs
living in usr/src. We need to use them from ports to record dependencies.

Discussed with: re(scottl)

build libthr on amd64.

Turn on the magic.

Enable libpthread build for powerpc

Hook libthread_db into the build, except for arm and powerpc.
Porting libthread_db to arm and/or powerpc is easy enough, but
we don't build gdb on those platforms yet.

Make bluetooth compile on all platforms

Reviewed by: imp, ru

Style fixes:
Main ones: mostly use conditional expressions in ifdefs instead of a
mixture of conditional expressions and nested ifdefs.
Nearby ones:
- don't do less than echo the code in the comment about libc_r
- fixed some internal insertion sort errors and indentation errors.

Fixed some insertion sort errors (external ones only).

Do not attempt to build libdisk, libthr and libc_r for arm.

Enable libdisk for powerpc build.

Put libypclnt behind NO_YP_LIBC

Hook libthr to the build for PowerPC

Moving forward, there's no reason to exclude libpthread on sparc64.

Connect libarchive to the build.

Add NO_BLUETOOTH knob to the build process

Requested by: phk
Reviewed by: imp (mentor), ru

The snmp_netgraph module depends on libnetgraph. So add a dependency and
add libnetgraph to the list of prebuilt libraries in the main Makefile.

Reviewed by: ru

Link libbsnmp to the build.

Link libngatm to the build (unless NOATM is set).

Update Bluetooth code.

Reviewed by: M. Warner Losh <imp@bsdimp.com>; John Hay <jhay@freebsd.org>
Approved by: M. Warner Losh <imp@bsdimp.com> (mentor)

- Support for multibyte charsets in LIBICONV.
- CD9660_ICONV, NTFS_ICONV and MSDOSFS_ICONV kernel options
(with corresponding modules).
- kiconv(3) for loadable charset conversion tables support.

Submitted by: Ryuichiro Imura <imura@ryu16.org>

Introduce more knobs to slim down FreeBSD userland

NO_TOOLCHAIN skips Compilers and Binutils
NO_USB skips USB stuff
NO_VINUM skips Vinum stuff
NO_ACPI skips ACPI stuff

Make build of libatm depend on existing NOATM conditional.

Imply NOLIBC_R for PowerPC.

Add libpthread to the alpha build.

Requested by ru: Since the majority of archs can now support the
build of libpthread, rearrange the Makefile to treat libpthread
as an exception.

Since it builds and seems to work OK, add libpthread to the amd64 build.