Remove gbde from rescue

cam: Add a XPORT_NVMF for NVMe over Fabrics sims

Reviewed by: ken, imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44713

rescue: add ipfw, pfctl

Rationale for this change:

- ipf is already there

- if the kernel is configured with a packet filter which drops packets
by default, pfctl or ipfw will be required in the rescue environment
to make the network functional.

- rescue's stated purpose is to be useful for small/embedded systems
(and is also quite useful for small jails); a rescue-based router
would want these binaries.

On amd64, this increases the size of rescue from 17346200 to 17907248,
or 561048 bytes (3.2%).

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1169

rescue: Stop using LDADD_zstd

Ideally we'd be able to use LDADD_foo here for all our various libs and
get the implicit dependencies, but rescue is a bit special. Historically
this was just used to pick up the "private" in the name automatically
(as far as I can tell), but now that _DP_pthread includes c we end up
pulling in a -lc from this (along with -lcompiler_rt and -lsys). This
results in -lc being before -lmd (and after, implictly, from the
compiler driver), which, for the specific situation here, results in
both libc.a's and libmd.a's md5c.o being included, giving duplicate
definitions of _libmd_MD5Init and other symbols. With LLD 16+ we
currently make that not an error for other reasons (which should
probably be fixed), but not for older versions, nor for BFD, and so the
build fails.

Fix all this by just using -lprivatezstd in place of LDADD_zstd, which
results in the exact same clang command line as we used to have prior to
adding c (and sys) to _DP_pthread when linking rescue. Note that bsdbox
already uses -lprivatezstd rather than LDADD_zstd.

This reverts commit 5fead429ebb3800fdd3fc0817d2c330b2a8d640f.

Reviewed by: imp
Fixes: 99ea67573164 ("lib{c,sys}: move auxargs more firmly into libsys")

rescue: Fix after zfsbootcfg addition

Get the library dependencies correct for zfsbootcfg. libzfsbootcfg
depends on a all of these...

Fixes: ac4847e6b0e9
Sponsored by: Netflix

rescue,nextboot: Install nextboot as a link to reboot, rm nextboot.sh

Reboot now emulates the nextboot shell script completely. Retire the
nextboot.sh script and install the link. Retain the same manual page,
since there's enough differences between nextboot and reboot that
talking about nextboot would likely be confusing in nextboot.8

The nextboot.sh script no longer exists, so doesn't need to be fixed up
to create rescue. However, now we need a link from nextboot to reboot.

Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43831
Differential Revision: https://reviews.freebsd.org/D43843

rescue: belatedly add zfsbootcfg

nextboot.sh uses zfsbootcfg to enable nextboot functionality for ZFS,
but zfsbootcfg was never added. Add it now since the nextboot binary
that replaced the script also uses it via system.

Sponsored by: Netflix
Reviewed by: kevans, kib, emaste
Differential Revision: https://reviews.freebsd.org/D43844

rescue: Don't explicitly link with libsys

libpthread contains the symbols we need when statically linked. This
was a leftover from a prior version of ef9871c6205c that I failed to
remove before I pushed.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D43782

libthr: move _umtx_op_err() to libsys

Declare in sys/umtx.h and implement in libsys. Explicitly link libthr
with libsys.

When building libthr static include _umtx_op_err so we don't break static
linkage with -lpthread.

Reviewed by: kib, emaste, imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/908

misc: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

This is for the misfits that have only a few: COPYRIGHT, gnu, tools,
rescue, and etc.

Sponsored by: Netflix

zfs: merge openzfs/zfs@804414aad

Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress

Obtained from: OpenZFS
OpenZFS commit: 804414aad224b432590afe3f9ec114ffb49e0f13

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Include bsd.linker.mk to get LINKER_TYPE and VERSION in rescue

To be able to use these make variables, you also have to include
bsd.linker.mk, apparently.

Fixes: 458f4722bf97
MFC after: 1 week

Suppress lld 16 errors about multiply defined symbols in rescue

lld >= 16 became more strict about multiply defined symbols. Since there
are many of those in crunchgen'd programs, turn off the check when
linking the rescue binary.

MFC after: 1 week

rescue: Fix link order of SSL libraries and fetch.

ld.bfd requires libraries to be linked in order. libssl requires
libcrypto. libfetch requires libssl. To fix the latter, move fetch
up above tar rather than listing the ssl libraries twice.

Reviewed by: delphij
Fixes: ea34aa4780e5 rescue: Add fetch(1) to the rescue tool.
Differential Revision: https://reviews.freebsd.org/D38304

rescue: Add fetch(1) to the rescue tool.

After a failed upgrade, having fetch(1) on a system that is
physically unnreachable would be very useful to download files
required to get the OS back up and functional.

On my system this adds 589,824 bytes (3.8%) to the binary size.

PR: 266224
Reported by: Dan Mahoney
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D38193

retire sconfig(8) ce(4)/cp(4) configuration tool

The ce(4) and cp(4) drivers have been retired.

Differential Revision: https://reviews.freebsd.org/D33469

rescue: Link libzutil after libzfs.

libzfs depends on symbols defined in libzutil. For static linking
ld.bfd is less forgiving than lld, so rescue was failing to link when
using ld.bfd due to unresolved symbols from libzutil used by libzfs.

Reviewed by: imp, kib, emaste
Differential Revision: https://reviews.freebsd.org/D36811

Fix conflict between libnvpair and libnv when building rescue binaries.

Submitted by: hselasky
Reviewed by: jhb
Sponsored by: NVIDIA Networking
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D32551

rescue: Link with -lncursesw instead of -lncursesw_real.

ld.bfd fails to link rescue with undefined symbol errors otherwise.

This reverts commit b158d4d7a12fad8e9c4509466d5f1ebd15862d9f.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D34123

rescue: include ping iff at least one of INET & INET6 is enabled

Ping does not build (and serves no purpose) if neither is enabled
(i.e., building WITHOUT_INET and WITHOUT_INET6).

This is equvalent to a4ef9e58bc0c but for rescue/.

PR: 260082
Fixes: a4ef9e58bc0c ("sbin: build ping if at least one of...")
Sponsored by: The FreeBSD Foundation

Add an internal libiscsiutil library.

Move some of the code duplicated between ctld(8) and iscsid(8) into a
libiscsiutil library.

Sharing the low-level PDU code did require having a
'struct connection' base class with a method table to permit separate
initiator vs target behavior (e.g. in handling proxy PDUs).

Reviewed by: mav, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33544

Retire synchronous PPP kernel driver sppp(4).

The last two drivers that required sppp are cp(4) and ce(4).

These devices are still produced and can be purchased
at Cronyx <http://cronyx.ru/hardware/wan.html>.

Since Roman Kurakin <rik@FreeBSD.org> has quit them, they no
longer support FreeBSD officially. Later they have dropped
support for Linux drivers to. As of mid-2020 they don't even
have a developer to maintain their Windows driver. However,
their support verbally told me that they could provide aid to
a FreeBSD developer with documentaion in case if there appears
a new customer for their devices.

These drivers have a feature to not use sppp(4) and create an
interface, but instead expose the device as netgraph(4) node.
Then, you can attach ng_ppp(4) with help of ports/net/mpd5 on
top of the node and get your synchronous PPP. Alternatively
you can attach ng_frame_relay(4) or ng_cisco(4) for HDLC.
Actually, last time I used cp(4) back in 2004, using netgraph(4)
instead of sppp(4) was already the right way to do.

Thus, remove the sppp(4) related part of the drivers and enable
by default the negraph(4) part. Further maintenance of these
drivers in the tree shouldn't be a big deal.

While doing that, remove some cruft and enable cp(4) compilation
on amd64. The ce(4) for some unknown reason marks its internal
DDK functions with __attribute__ fastcall, which most likely is
safe to remove, but without hardware I'm not going to do that, so
ce(4) remains i386-only.

Reviewed by: emaste, imp, donner
Differential Revision: https://reviews.freebsd.org/D32590
See also: https://reviews.freebsd.org/D23928

rescue: fix after with to a ldscript of libncursesw.a

Fix building rescue/rescue when sanitizers are enabled

We have to ensure that we don't link any instrumented object files
into rescue as it is a static executable and static binaries can't
use the sanitizer runtime.

Reviewed By: imp
Differential Revision: https://reviews.freebsd.org/D31044

ping: add a ping6 hard link for backwards compatibility

When invoked as "ping6", ping will now attempt to use ICMPv6 for hostnames
that resolve both IPv4 and IPv6 addresses.

Reviewed by: bz, manu
MFC-With: r368045
Differential Revision: https://reviews.freebsd.org/D27384

Merge ping6 to ping

There is now a single ping binary, which chooses to use ICMP or ICMPv4
based on the -4 and -6 options, and the format of the address.

Submitted by: Ján Sučan <sucanjan@gmail.com>
Sponsored by: Google LLC (Google Summer of Code 2019)
MFC after: Never
Differential Revision: https://reviews.freebsd.org/D21377

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

libifconfig now depends on libm due to usage of log10().

ld.bfd in particular requires -lm to come after libifconfig on the
command line when linking rescue.

Reviewed by: freqlabs, adrian
Differential Revision: https://reviews.freebsd.org/D26258

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

Move ifconfig SFP status functionality into libifconfig

libifconfig_sfp.h provides an API in libifconfig for querying SFP module
properties, operational status, and vendor strings, as well as descriptions
of the various fields, string conversions, and other useful helpers for
implementing user interfaces.

SFP module status is obtained by reading registers via an I2C interface.
Descriptions of these registers and the values therein have been collected
in a Lua table which is used to generate all the boilerplace C headers and
source files for accessing these values, their names, and descriptions.
The generated code is fully commented and readable.

This is the first use of libifconfig in ifconfig itself. For now, the
scope remains very limited. Over time, more of ifconfig will be replaced
with libifconfig.

Some minor changes to the formatting of ifconfig output have been made:
- Module memory hex dumps are indented one extra space as a result of using
hexdump(3) instead of a bespoke hex dump function.
- Media descriptions have an added two-character short-name in parenthesis.
- QSFP modules were incorrectly displaying TX bias current as power. Now
TX channels display bias current, and this change has been made for both
SFP and QSFP modules for consistency.

A Lua binding for libifconfig including this functionality is implemented
but has not been included in this commit. The plan is for it to be
committed after dynamic module loading has been enabled in flua.

Reviewed by: kp, melifaro
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D25494

- Cleanups related to sparc64 removal.
- Remove remains of sparc64 files.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D25831

rescue: Remove useless linking with libl

Remove unused defines since r147075

When the OpenBSD dhclient was brought in 14 years ago, we stopped
supporting building a reduced sized dhclient, yet retained the options
here. Also, the OpenBSD dhclient doesn't need lint defined, so it can
go too.

rescue: set NO_SHARED in Makefile

The rescue binary is built statically via the Makefile generated by
crunchgen, but that does not trigger other shared/static logic in
bsd.prog.mk - in particular disabling retpolineplt with static linking.

PR: 233336
Reported by: Charlie Li
Sponsored by: The FreeBSD Foundation

rescue: Add bectl(8)

Requested by: kevans, Shawn Webb
MFC after: 3 days

Avoid referencing private lib names directly.

Sponsored by: Dell EMC Isilon

rescue ipf: Remove hacks and link in libipf directly.

Sponsored by: Dell EMC Isilon

Remove badsect(8).

Failure modes of the modern (that is, produced in the last 25 years)
hard drives and SSDs made the utility outdated. Since the kernel
interface to support it was removed in r324853, cut the userspace
remnants as well.

Discussed with: bde (who does not like the removal)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Remove rcmds.

If they are still needed, you can find them in the net/bsdrcmds port.

This was proposed June, 20th and approved by various committers [1].
They have been marked as deprecated on CURRENT in r320644 [2] on July, 4th.
Both stable/11 and release/11.1 contain the deprecation notice (thanks to
allanjude@).

Note that ruptime(1)/rwho(1)/rwhod(8) were initially thought to be part of
rcmds but this was a mistake and those are therefore NOT removed.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html
[2] https://svnweb.freebsd.org/base?view=revision&revision=320644

Reviewed by: bapt, brooks
Differential Revision: https://reviews.freebsd.org/D12573

Add zstd to the rescue binary

Having zstd might be useful in rescue to be able to access to log files
compressed by zstandard

Suggested by: ian

More ATM and NATM removal

Submitted by: ak
Reviewed by: ngie
Differential Revision: https://reviews.freebsd.org/D10511

Remove NATM configuration bits and assorted NATM and ATM remnants.

Reported by: ak
Reviewed by: ngie (first version)
Differential Revision: https://reviews.freebsd.org/D10497

Remove the NATM framework including the en(4), fatm(4), hatm(4), and
patm(4) devices.

Maintaining an address family and framework has real costs when we make
infrastructure improvements. In the case of NATM we support no devices
manufactured in the last 20 years and some will not even work in modern
motherboards (some newer devices that patm(4) could be updated to
support apparently exist, but we do not currently have support).

With this change, support remains for some netgraph modules that don't
require NATM support code. It is unclear if all these should remain,
though ng_atmllc certainly stands alone.

Note well: FreeBSD 11 supports NATM and will continue to do so until at
least September 30, 2021. Improvements to the code in FreeBSD 11 are
certainly welcome.

Reviewed by: philip
Approved by: harti

Fix linking /rescue/rescue to multiple programs in usr.bin after r315113

I meant for the line that conditionally added in /usr/bin/nc support to
be `+=', not `=`. This restores hardlinks for all programs in usr.bin
specified before nc(1), e.g., bunzip2 and tar.

Reported by: Andrzej Tobola <ato@iem.pw.edu.pl>
Submitted by: Scott Allendorf <scott-allendorf@uiowa.edu>
Pointyhat to: ngie
MFC after: 3 days
Sponsored by: Dell EMC Isilon

Make rescue use SRCTOP

Transition to SRCTOP. Also, standardize of {} instead of () variable
deliminators while I'm here.

Differential Revision: https://reviews.freebsd.org/D9932
Sponsored by: Netflix
Silence On: arch@ (twice)

Conditionally compile [additional] programs into rescue(8) if requested

Trivial oversight missed in r314240 cleanup because I enable these knobs on
my test machines.

MK_INET6_SUPPORT - rtsol
MK_NETCAT - nc

MFC after: 3 days
Sponsored by: Dell EMC Isilon

Conditionally compile certain programs into rescue(8) if requested

MK_CCD - ccdconfig
MK_ROUTED - routed, rtquery

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Add shutdown/poweroff support to rescue(8)

shutdown is a safer way to power off than reboot (in general), because of
the added shutdown process that it executes via /etc/rc.shutdown . It was
odd that it was missing from rescue(8) since reboot and friends were
added in past commits.

While here, alias poweroff to shutdown for parity with sbin/shutdown/Makefile

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes: yes

Split /rescue into its own package.

Sponsored by: The FreeBSD Foundation

Fix the build after ifconfig was converted over to lib80211 in r291470

Reported by: jenkins, O. Hartmann <ohartman@zedat.fu-berlin.de>
Pointyhat to: adrian

Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles(). It drops the
unused 'swapfile' argument and adds a new function pointer argument for
a symbol resolving function. Native kernels still use _fdnlist() from
libc to resolve symbols if a resolver function is not supplied, but cross
kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
Each backend is responsible for implementing kvm_read2() for a given
vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
powerpc cores).
- Add internal helper routines for the common page offset hash table used
by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
to be cross-friendly by using private constants instead of ones that
vary by platform (e.g. PAGE_SIZE). Static assertions are present when
a given backend is compiled natively to ensure the private constants
match the real ones.
- Enable all of the existing vmcore backends on all platforms. This means
that libkvm on any platform should be able to perform KVA translation
and read data from a vmcore of any platform.

Tested on: amd64, i386, sparc64 (marius)
Differential Revision: https://reviews.freebsd.org/D3341

Make iscsictl and iscsid build if MK_ISCSI == yes

MFC after: 1 month
X-MFC with: r289452

Add iscsictl(8) and iscsid(8) to rescue(8). The point is to make it
easier to build md_root images from rescue(8), to use with iSCSI boot.

The change increases the size of rescue by 62kB, from 8728kB to 8790kB.

Reviewed by: bapt@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3865

Significant upgrades to sa(4) and mt(1).

The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.

o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.

o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.

This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.

For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.

o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.

o New, extensible tape driver parameter get/set interface.

o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.

o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.

o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.

o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.

o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.

o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.

This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.

rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.

src/share/man/man4/sa.4
Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.

Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.

Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.

Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.

Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.

Add a new MTIOCRBLIM ioctl to get block limits information.

Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().

scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.

Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.

For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.

Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.

Add a partition number to the sa(4) softc.

Improve device departure handling. The previous implementation
led to hangs when the device was open.

If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.

So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.

Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.

Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().

Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.

Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.

This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.

However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.

The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.

This includes definitions for the following new ioctls:

MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */

usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.

Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.

The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.

The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.

Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.

Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.

Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.

Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.

Sponsored by: Spectra Logic
MFC after: 1 month

Merge xz 5.2.0.

This brings support for multi-threaded compression. This brings close
N times faster compression where N is the number of CPU cores.
Because of this, liblzma now depends on libthr.

Soon libarchive will be modified to use the new lzma API.

Thanks to antoine@ for the exp-run.

Differential Revision: https://reviews.freebsd.org/D1786
Reviewed by: bapt

Only build vi support into rescue if MK_VI != no

Sponsored by: EMC / Isilon Storage Division

Add sleep(1) to /rescue. This adds 664 bytes to the binary on amd64 but
it's pretty useful for shell scripts.

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Add libxo, now needed by df(1).

Pointed out by: rodrigc@ (thanks!)

Remove duplicate prog.

Add zdb into rescue environment.

On amd64, this would increase the binary size by 1.1MiB and
make it possible to examine zpool status offline, useful for
recovery and diagnostic purposes.

Submitted by: sef
Obtained from: FreeNAS
MFC after: 2 weeks

In r232153, libarchive 3.0.3 was imported, replacing the archive_hash.h
header with archive_crypto_private.h, and its ARCHIVE_HASH_xxx macros
were renamed to ARCHIVE_CRYPTO_xxx.

Rename these macros in lib/libarchive/config_freebsd.h, to re-enable the
hashes for libarchive again. This affects the mtree format writer, and
the xar format reader and writer modules.

This also requires changes in the library order for statically linking
rescue, otherwise ld would complain about redefined symbols. Thanks to
jkim for pointing out the solution.

Reviewed by: kientzle
MFC after: 1 week

Replace all uses of libncurses and libtermcap with their wide character
variants. This allows usable file system images (i.e. those with both a
shell and an editor) to be created with only one copy of the curses library.

Exp-run: antoine
PR: 189842
Discussed with: bapt
Sponsored by: DARPA, AFRL

Remove ia64.

This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation

This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h

Discussed at: BSDcan

Minor style nits...

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

- Fix build WITHOUT_ZFS/WITHOUT_CDDL after r265229, bin/ps needs libjail.

MFC after: 2 weeks
X-MFC-with: r265229

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

Remove IPX support.

IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.

Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.

- Trim an unused and bogus Makefile for mount_smbfs.
- Reconnect with some minor modifications, in particular now selsocket()
internals are adapted to use sbintime units after recent'ish calloutng
switch.

Add less to rescue build. On amd64, this increases rescue size by about
130KB or 2.4%.

MFC after: 1 month

Remove all legacy ATA code parts, not used since options ATA_CAM enabled in
most kernels before FreeBSD 9.0. Remove such modules and respective kernel
options: atadisk, ataraid, atapicd, atapifd, atapist, atapicam. Remove the
atacontrol utility and some man pages. Remove useless now options ATA_CAM.

No objections: current@, stable@
MFC after: never

WiP merge of libzfs_core (MFV r238590, r238592)
not yet working, ioctl handling needs to be changed

Disconnect non-MPSAFE SMBFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netsmb, which is a base
requirement for SMBFS.

In the while SMBFS regular users can use FUSE interface and smbnetfs
port to work with their SMBFS partitions.

Also, there are ongoing efforts by vendor to support in-kernel smbfs,
so there are good chances that it will get relinked once properly locked.

This is not targeted for MFC.

Disconnect non-MPSAFE NTFS from the build in preparation for dropping
GIANT from VFS. This code is particulary broken and fragile and other
in-kernel implementations around, found in other operating systems,
don't really seem clean and solid enough to be imported at all.
If someone wants to reconsider in-kernel NTFS implementation for
inclusion again, a fair effort for completely fixing and cleaning it
up is expected.

In the while NTFS regular users can use FUSE interface and ntfs-3g
port to work with their NTFS partitions.

This is not targeted for MFC.

Disconnect non-MPSAFE NWFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netncp, which is a base
requirement for NWFS.

In the possibility of a future maintenance of the code and later
readd to the FreeBSD base, maybe we should think about a better location
for netncp. I'm not entirely sure the / top location is actually right,
however I will let network people to comment on that more specifically.

This is not targeted for MFC.

unbreak build: rescue now needs -ljail if MK_ZFS is on

Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after: 2 weeks

Disable jail support in ifconfig when either building a rescue
image or MK_JAIL knob has been set to "no".

Reviewed by: bz
Approved by: adrian (mentor)

-lreadline is not required anymore.

-lpthread is required by -lzfs so should be later in LIBS list.

There were no "undefined symbol pthread_xxx" errors during the link before
this fix only because of STATIC_LIB_REQUIRE() declarations in
lib/libthr/thread/thr_init.c.

Add netcat (nc) to /rescue.

MFC after: 3 weeks

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

Break out the rules which generate crunchgen'ed binaries into a separate
.mk file so they can be reused.

Introduce a new option, CRUNCH_BUILDTOOLS, which lists the binaries that
require tools built in the local architecture. sh and csh both require this.
It was previously hardcoded in rescue/rescue/Makefile .

Introduce a new option, CRUNCH_SHLIBS, which lists the shared libraries
to link against. These override the static libraries listed in CRUNCH_LIBS.
Some build environments may wish to use a handful of shared libraries
(eg libc.so) so other small, dynamic binaries can be run in the environment.

Remove the now-shared code from rescue/rescue/Makefile and introduce the
CRUNCH_BUILDTOOLS option for the above shells.

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

Enable liblzma support in libarchive
Adjust dependencies for programs using libarchive
Add xz and linkage against liblzma to rescue system

Approved by: kientzle, delphij (mentor)
MFC after: 2 weeks

Remove libulog from the bootstrap again.

libulog now only provides functions that are used by various packages
from the ports tree, namely the libutempter ones. There is no reason to
link it into the crunch/fixit binaries anymore.

Unbreak rescue(8). We should also link against libulog now.

Add the "vnet" and "-vnet" options, to allow moving interfaces between
jails with VIMAGE.

Approved by: bz (mentor)

Add zfs/zpool to rescue programs

PR: bin/125878
Submitted by: nork@
MFC after: 3 days

Libarchive is sprouting dependencies on libmd and libcrypto.
Because crunchgen drops any repeated library (keeping only the
first), the -lcrypto reference must be moved to after -larchive,
not merely duplicated.

I'm considering changing crunchgen's handling of duplicate
libraries, but that's a rather more delicate issue.

*** empty log message ***

Remove if_ppp(4) and if_sl(4).

Not only did these two drivers depend on IFF_NEEDSGIANT, they were
broken 7 months ago during the MPSAFE TTY import. if_ppp(4) has been
replaced by ppp(8). There is no replacement for if_sl(4).

If we see regressions in for example the ports tree, we should just use
__FreeBSD_version 800045 to check whether if_ppp(4) and if_sl(4) are
present. Version 800045 is used to denote the import of MPSAFE TTY.

Discussed with: rwatson, but also rwatson's IFF_NEEDSGIANT emails on the
lists.

Fix build when WITH_SSP is set explicitly.

Submitted by: Jeremie Le Hen

Take a moment to tidy some white space while I'm here. No functional
changes for this commit.

We don't need pax and tar. These days tar is a strict superset of
pax. Per discssuion on arch@ eliminate it.

pkill(1) first appeared in /usr/bin, but later it was moved to /bin
for the convenience of rc.d. Now it has happily lived there for quite
a while. So move the pkill(1) source files from usr.bin to bin, too.

Approved by: gad

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Remove netatm from HEAD as it is not MPSAFE and relies on the now removed
NET_NEEDS_GIANT. netatm has been disconnected from the build for ten
months in HEAD/RELENG_7. Specifics:

- netatm include files
- netatm command line management tools
- libatm
- ATM parts in rescue and sysinstall
- sample configuration files and documents
- kernel support as a module or in NOTES
- netgraph wrapper nodes for netatm
- ctags data for netatm.
- netatm-specific device drivers.

MFC after: 3 weeks
Reviewed by: bz
Discussed with: bms, bz, harti

Adding glabel alias killed gpart alias; fix it.

Add an alias for glabel(8).

o Build geom for all platforms.
o Don't build bsdlabel for ia64.
o Don't build fdisk and gpt for ia64.

Add some essential tools to rescue(8) in order to make it
a versatile emergency tool:

o sed(1) as a multi-purpose text filter -- can do grep's job and much more.
o head(1), tail(1), and tee(1) as idiomatic text filters.
o mt(1) to control tape drives (PR misc/98383).
o chown(8) aka chgrp(8) to complement the ch* subset.
o pkill(1) aka pgrep(1) to control running processes easily and thus to be
able to recover from a serious problem or a fatal typo in an otherwise live
system w/o a reboot. (It also deserves adding to rescue(8) for its having
triggered a latent bug in crunchgen(1), but we had better add a regression
test for that. :-)

The resulting change in rescue(8) size has the following order of magnitude
on i386: 3787656 - 3727872 = 59784, i.e. just a tad.

Discussed on: -hackers (I seem to have wearied all opponents :-)
PR: misc/98383

Disconnect netatm from the build as it is not MPSAFE and relies on
NET_NEEDS_GIANT, which will shortly be removed. This is done in a
away that it may be easily reattached to the build before 7.1 if
appropriate locking is added. Specifics:

- Don't install netatm include files
- Disconnect netatm command line management tools
- Don't build libatm
- Don't include ATM parts in rescue or sysinstall
- Don't install sample configuration files and documents
- Don't build kernel support as a module or in NOTES
- Don't build netgraph wrapper nodes for netatm

This removes the last remaining consumer of NET_NEEDS_GIANT.

Reviewed by: harti
Discussed with: bz, bms
Approved by: re (kensmith)

- Remove UMAP filesystem. It was disconnected from build three years ago,
and it is seriously broken.

Discussed on: freebsd-arch@
Approved by: re (mux)

Replace the GNU gzip with a slightly modified NetBSD gzip. The
NetBSD version is a feature-to-feature re-implementation of GNU
gzip using the freely-redistributable zlib and this version is
expected to be mostly bug-to-bug compatible with the GNU
implementation.

- Because this is a piece of mature code and we want to make
changes so it is added directly rather than importing to
src/contrib.
- Connect newly added code to src/usr.bin/ and rescue/rescue
build.
- Disconnect the GNU gzip code from build for now, they will
be eventually removed completely.
- Provide two new src.conf(5) knobs, WITHOUT_BZIP2_SUPPORT and
WITHOUT_BZIP2.

Tested by: kris (full exp-7 pointyhat build)
Approved by: core (importing a 4-clause BSD licensed file)
Approved by: re (adding new utility during -HEAD code slush)

Remove mount_ext2fs.

Remove references to mount_devfs, mount_fdescfs, mount_linprocfs, mount_procfs.

Reminded by: ru

Remove alpha left-overs.

Pass -DCRUNCH down to standard targets in individual makefiles.

*sigh*
Move the -lbsdxml after -lgeom, so that ld doesn't get confused and pretend
he can't find the symbol from libbsdxml needed in libgeom.
This should fix the rescue build breakage.

Extend coverage of the MK_IPX build option to the following:

- <netipx> headers [1]
- IPX library (libipx)
- IPX support in ifconfig(8)
- IPXrouted(8)
- new MK_NCP option

New MK_NCP build option controls:

- <netncp> and <fs/nwfs> headers
- NCP library (libncp)
- ncplist(1) and ncplogin(1)
- mount_nwfs(8)
- ncp and nwfs kernel modules

User knobs: WITHOUT_IPX, WITHOUT_IPX_SUPPORT, WITHOUT_NCP.

[1] <netsmb/netbios.h> unconditionally uses <netipx> headers
so they are still installed. This needs to be dealt with.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Install nextboot in /rescue as /rescue/nextboot rather than
/rescue/nextboot.sh to match the name in /sbin (/sbin/nextboot).

Reviewed by: gtetlow
MFC after: 1 week

Fix path to dhclient-script and reconnect to build.

Temporarily disconnect dhclient from the build while I import the
OpenBSD version.

Patches from Ruslam Ermilov to remove NetBSD bits from Makefiles and cleanup
build problems with rescue.

Run "make obj" before "make build-tools" in the directories used for
building the rescue binary. This fixes a problem with NO_TCSH, where
the "make obj" stage of buildworld doesn't recurse into bin/csh,
resulting in csh build-tools being put into /usr/src/bin/csh.

Pointed out by: dougb (on hackers@)

Wrap BSD r* commands in NO_RCMDS.
Change NO_RCMDNS to NO_RCMDS and do the switch in bsd.compat.mk.

Discussed with: ru, nectar

Add chroot. It can be a real lifesaver, and adds less than 2 kB.

Ignoring MAKEFLAGS in rev. 1.15 was a very bad idea. This causes
wrong share/mk files to be used, which can be fatal with upgrades
or downgrades, e.g., when building RELENG_5 on HEAD.

Reported by: glebius

For now, just exclude -P from MAKEFLAGS when running crunchgen(1).
(Note that it will still break when run with certain -d options.)

The real solution is to fix make(1) to not print stuff on stdout
when it's not supposed to, e.g., through the -P and -dX options,
and to fix crunchgen(1) to not redirect stderr to stdout when
running make(1). Once this is implemented, this hack can go.

Don't call "objs" target in rescue.mk twice.

Make "===> " prefixes look sane.

NOINET6 -> NO_INET6

NOCRYPT -> NO_CRYPT

NOATM -> NO_ATM

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Add missing dependencies of $(OUTPUTS) on source makefiles.

Caught by: make -DNOCLEAN buildworld

Replace GNU tar with BSD tar.

Remove references to NO_VINUM

For variables that are only checked with defined(), don't provide
any fake value.

As with the non-rescue version don't build fore_dnld when NOATM is defined.

Remove dangling raidctl reference

Add id(1) (aka groups(1) aka whoami(1)) since it is used by install.sh.

Revision 1.7 of this file added information
about the location of each program's source.

This update optimizes the build a bit by giving that
information to crunchgen rather than asking crunchgen
to do a directory search to locate sources.

Approved by: gordon (Mentor)

Properly spell make binary as ${MAKE}.

This fixes buildworld for systems with old make(1) binaries.

Reported by: Benjamin Close

Remove cxconfig and add sconfig

# maybe we can remove sconfig later from rescue?

Submitted by: Roman Kurakin <rik@cronyx.com>
Approved by: re@ <scottl>

Reconnect ipfstat, ipnat, and ipftest to the build now that if_xname
support is enabled.

Approved by: re (scottl)

Alternate version of rev 1.20.
Comment out rather than totally remove the ipfilter pieces that we need
reconnected some day. This is now only ipnat as it is for configuring NAT.
ipfstat is meant for reporting statistics/filter lists. For /rescue it is
enough to configure lists but not view the installed ones.

Temporarily disable ipfstat and ipnat in /rescue to fix world.

No need to check for the directory, fols are assumed to have all the
correct sources. Only check the crypto macros.

- Support for multibyte charsets in LIBICONV.
- CD9660_ICONV, NTFS_ICONV and MSDOSFS_ICONV kernel options
(with corresponding modules).
- kiconv(3) for loadable charset conversion tables support.

Submitted by: Ryuichiro Imura <imura@ryu16.org>

Teach rescue about NOATM, NO_VINUM and NOINET6

Build the atmconfig utility into rescue.

When using crunchgen, blow away MAKEFLAGS. This fixes make -j <#> -P
from passing the -P flag to crunchgen which seems to confuse crunchgen
horribly. This is the preferable solution to modifing crunchgen to
unset the MAKEFLAGS environment variable.

Submitted by: gad@

Fix rescue build using -j. The problem appears to be make not being able
to find the source when the object was specified as <directory>/foo.o.

The fix makes the build go through a make objs before compiling the rest
of the crunchgen. This ensures that the dhclient bits are built in the
correct place where they are picked up for the final compile of rescue.

I'd like to thank dwhite@ and gad@ for helping me track down the problem.

Fast testing box provided by: phk@ (thanks)

Missed one, remove one more.

Trim /rescue.

Discussed on: freebsd-arch@

Obey NOCRYPT

Submitted by: Skip Ford <skip.ford@verizon.net>

NO_TCSH would be set in /etc/make.conf, not here. Also obey NO_IPFILTER.

consistify the style some

Remove the commented out 'rmail' before someone gets a really bad idea.

Depessimize rescue build by only clean'ing and obj'ing the directories we
care about, not a full usr.bin tree. This should reduce buildworld times
pretty drastically.

Requested by: lots of people

Turn rescue back on, conditional to NORESCUE. We seem to be split on
using underscores or not, so I just randomly picked a style. I think
I have the logic correct, but if someone wants to give it a once over
that would be good.

Tim submitted a patch to fix the cross-building issues which I tested
with a tinderbox run for sparc64.

Submitted by: Tim Kientzle <kientzle@acm.org>

Remove mount_portalfs, it's just wrong in this context.

Don't build mount_nwfs or mount_smbfs in rescue.
Build fdisk_pc98 on pc98 arch, not fdisk.
Don't alias disklabel on pc98, ia64.
Don't build fdisk on sparc64, alpha.

Pointed out by: tmm@
Submitted by: Tim Kientzle <kientzle@acm.org>

Fix fdisk naming issues on pc98 and other platforms.
Fix disklabel, bsdlabel, and sunlabel on various platforms.

Noticed by: tmm@
Submitted by: Tim Kientzle <kientzle@acm.org>

Fix typo sparc -> sparc64

Submitted by: tmm@

Add /rescue bits. This basically encompasses all of bin and sbin along
with a couple of bits from usr.bin in a crunchgen'd binary.

Submitted by: Tim Kientzle <kientzle@acm.org>

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Include bsd.linker.mk to get LINKER_TYPE and VERSION in rescue

To be able to use these make variables, you also have to include
bsd.linker.mk, apparently.

Fixes: 458f4722bf97
MFC after: 1 week

Suppress lld 16 errors about multiply defined symbols in rescue

lld >= 16 became more strict about multiply defined symbols. Since there
are many of those in crunchgen'd programs, turn off the check when
linking the rescue binary.

MFC after: 1 week

rescue: Fix link order of SSL libraries and fetch.

ld.bfd requires libraries to be linked in order. libssl requires
libcrypto. libfetch requires libssl. To fix the latter, move fetch
up above tar rather than listing the ssl libraries twice.

Reviewed by: delphij
Fixes: ea34aa4780e5 rescue: Add fetch(1) to the rescue tool.
Differential Revision: https://reviews.freebsd.org/D38304

rescue: Add fetch(1) to the rescue tool.

After a failed upgrade, having fetch(1) on a system that is
physically unnreachable would be very useful to download files
required to get the OS back up and functional.

On my system this adds 589,824 bytes (3.8%) to the binary size.

PR: 266224
Reported by: Dan Mahoney
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D38193

retire sconfig(8) ce(4)/cp(4) configuration tool

The ce(4) and cp(4) drivers have been retired.

Differential Revision: https://reviews.freebsd.org/D33469

rescue: Link libzutil after libzfs.

libzfs depends on symbols defined in libzutil. For static linking
ld.bfd is less forgiving than lld, so rescue was failing to link when
using ld.bfd due to unresolved symbols from libzutil used by libzfs.

Reviewed by: imp, kib, emaste
Differential Revision: https://reviews.freebsd.org/D36811

Fix conflict between libnvpair and libnv when building rescue binaries.

Submitted by: hselasky
Reviewed by: jhb
Sponsored by: NVIDIA Networking
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D32551

rescue: Link with -lncursesw instead of -lncursesw_real.

ld.bfd fails to link rescue with undefined symbol errors otherwise.

This reverts commit b158d4d7a12fad8e9c4509466d5f1ebd15862d9f.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D34123

rescue: include ping iff at least one of INET & INET6 is enabled

Ping does not build (and serves no purpose) if neither is enabled
(i.e., building WITHOUT_INET and WITHOUT_INET6).

This is equvalent to a4ef9e58bc0c but for rescue/.

PR: 260082
Fixes: a4ef9e58bc0c ("sbin: build ping if at least one of...")
Sponsored by: The FreeBSD Foundation

Add an internal libiscsiutil library.

Move some of the code duplicated between ctld(8) and iscsid(8) into a
libiscsiutil library.

Sharing the low-level PDU code did require having a
'struct connection' base class with a method table to permit separate
initiator vs target behavior (e.g. in handling proxy PDUs).

Reviewed by: mav, emaste
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33544

Retire synchronous PPP kernel driver sppp(4).

The last two drivers that required sppp are cp(4) and ce(4).

These devices are still produced and can be purchased
at Cronyx <http://cronyx.ru/hardware/wan.html>.

Since Roman Kurakin <rik@FreeBSD.org> has quit them, they no
longer support FreeBSD officially. Later they have dropped
support for Linux drivers to. As of mid-2020 they don't even
have a developer to maintain their Windows driver. However,
their support verbally told me that they could provide aid to
a FreeBSD developer with documentaion in case if there appears
a new customer for their devices.

These drivers have a feature to not use sppp(4) and create an
interface, but instead expose the device as netgraph(4) node.
Then, you can attach ng_ppp(4) with help of ports/net/mpd5 on
top of the node and get your synchronous PPP. Alternatively
you can attach ng_frame_relay(4) or ng_cisco(4) for HDLC.
Actually, last time I used cp(4) back in 2004, using netgraph(4)
instead of sppp(4) was already the right way to do.

Thus, remove the sppp(4) related part of the drivers and enable
by default the negraph(4) part. Further maintenance of these
drivers in the tree shouldn't be a big deal.

While doing that, remove some cruft and enable cp(4) compilation
on amd64. The ce(4) for some unknown reason marks its internal
DDK functions with __attribute__ fastcall, which most likely is
safe to remove, but without hardware I'm not going to do that, so
ce(4) remains i386-only.

Reviewed by: emaste, imp, donner
Differential Revision: https://reviews.freebsd.org/D32590
See also: https://reviews.freebsd.org/D23928

rescue: fix after with to a ldscript of libncursesw.a

Fix building rescue/rescue when sanitizers are enabled

We have to ensure that we don't link any instrumented object files
into rescue as it is a static executable and static binaries can't
use the sanitizer runtime.

Reviewed By: imp
Differential Revision: https://reviews.freebsd.org/D31044

ping: add a ping6 hard link for backwards compatibility

When invoked as "ping6", ping will now attempt to use ICMPv6 for hostnames
that resolve both IPv4 and IPv6 addresses.

Reviewed by: bz, manu
MFC-With: r368045
Differential Revision: https://reviews.freebsd.org/D27384

Merge ping6 to ping

There is now a single ping binary, which chooses to use ICMP or ICMPv4
based on the -4 and -6 options, and the format of the address.

Submitted by: Ján Sučan <sucanjan@gmail.com>
Sponsored by: Google LLC (Google Summer of Code 2019)
MFC after: Never
Differential Revision: https://reviews.freebsd.org/D21377

loader: zfs should support bootonce an nextboot

bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512

libifconfig now depends on libm due to usage of log10().

ld.bfd in particular requires -lm to come after libifconfig on the
command line when linking rescue.

Reviewed by: freqlabs, adrian
Differential Revision: https://reviews.freebsd.org/D26258

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

Move ifconfig SFP status functionality into libifconfig

libifconfig_sfp.h provides an API in libifconfig for querying SFP module
properties, operational status, and vendor strings, as well as descriptions
of the various fields, string conversions, and other useful helpers for
implementing user interfaces.

SFP module status is obtained by reading registers via an I2C interface.
Descriptions of these registers and the values therein have been collected
in a Lua table which is used to generate all the boilerplace C headers and
source files for accessing these values, their names, and descriptions.
The generated code is fully commented and readable.

This is the first use of libifconfig in ifconfig itself. For now, the
scope remains very limited. Over time, more of ifconfig will be replaced
with libifconfig.

Some minor changes to the formatting of ifconfig output have been made:
- Module memory hex dumps are indented one extra space as a result of using
hexdump(3) instead of a bespoke hex dump function.
- Media descriptions have an added two-character short-name in parenthesis.
- QSFP modules were incorrectly displaying TX bias current as power. Now
TX channels display bias current, and this change has been made for both
SFP and QSFP modules for consistency.

A Lua binding for libifconfig including this functionality is implemented
but has not been included in this commit. The plan is for it to be
committed after dynamic module loading has been enabled in flua.

Reviewed by: kp, melifaro
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D25494

- Cleanups related to sparc64 removal.
- Remove remains of sparc64 files.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D25831

rescue: Remove useless linking with libl

Remove unused defines since r147075

When the OpenBSD dhclient was brought in 14 years ago, we stopped
supporting building a reduced sized dhclient, yet retained the options
here. Also, the OpenBSD dhclient doesn't need lint defined, so it can
go too.

rescue: set NO_SHARED in Makefile

The rescue binary is built statically via the Makefile generated by
crunchgen, but that does not trigger other shared/static logic in
bsd.prog.mk - in particular disabling retpolineplt with static linking.

PR: 233336
Reported by: Charlie Li
Sponsored by: The FreeBSD Foundation

rescue: Add bectl(8)

Requested by: kevans, Shawn Webb
MFC after: 3 days

Avoid referencing private lib names directly.

Sponsored by: Dell EMC Isilon

rescue ipf: Remove hacks and link in libipf directly.

Sponsored by: Dell EMC Isilon

Remove badsect(8).

Failure modes of the modern (that is, produced in the last 25 years)
hard drives and SSDs made the utility outdated. Since the kernel
interface to support it was removed in r324853, cut the userspace
remnants as well.

Discussed with: bde (who does not like the removal)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Remove rcmds.

If they are still needed, you can find them in the net/bsdrcmds port.

This was proposed June, 20th and approved by various committers [1].
They have been marked as deprecated on CURRENT in r320644 [2] on July, 4th.
Both stable/11 and release/11.1 contain the deprecation notice (thanks to
allanjude@).

Note that ruptime(1)/rwho(1)/rwhod(8) were initially thought to be part of
rcmds but this was a mistake and those are therefore NOT removed.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html
[2] https://svnweb.freebsd.org/base?view=revision&revision=320644

Reviewed by: bapt, brooks
Differential Revision: https://reviews.freebsd.org/D12573

Add zstd to the rescue binary

Having zstd might be useful in rescue to be able to access to log files
compressed by zstandard

Suggested by: ian

More ATM and NATM removal

Submitted by: ak
Reviewed by: ngie
Differential Revision: https://reviews.freebsd.org/D10511

Remove NATM configuration bits and assorted NATM and ATM remnants.

Reported by: ak
Reviewed by: ngie (first version)
Differential Revision: https://reviews.freebsd.org/D10497

Remove the NATM framework including the en(4), fatm(4), hatm(4), and
patm(4) devices.

Maintaining an address family and framework has real costs when we make
infrastructure improvements. In the case of NATM we support no devices
manufactured in the last 20 years and some will not even work in modern
motherboards (some newer devices that patm(4) could be updated to
support apparently exist, but we do not currently have support).

With this change, support remains for some netgraph modules that don't
require NATM support code. It is unclear if all these should remain,
though ng_atmllc certainly stands alone.

Note well: FreeBSD 11 supports NATM and will continue to do so until at
least September 30, 2021. Improvements to the code in FreeBSD 11 are
certainly welcome.

Reviewed by: philip
Approved by: harti

Fix linking /rescue/rescue to multiple programs in usr.bin after r315113

I meant for the line that conditionally added in /usr/bin/nc support to
be `+=', not `=`. This restores hardlinks for all programs in usr.bin
specified before nc(1), e.g., bunzip2 and tar.

Reported by: Andrzej Tobola <ato@iem.pw.edu.pl>
Submitted by: Scott Allendorf <scott-allendorf@uiowa.edu>
Pointyhat to: ngie
MFC after: 3 days
Sponsored by: Dell EMC Isilon

Make rescue use SRCTOP

Transition to SRCTOP. Also, standardize of {} instead of () variable
deliminators while I'm here.

Differential Revision: https://reviews.freebsd.org/D9932
Sponsored by: Netflix
Silence On: arch@ (twice)

Conditionally compile [additional] programs into rescue(8) if requested

Trivial oversight missed in r314240 cleanup because I enable these knobs on
my test machines.

MK_INET6_SUPPORT - rtsol
MK_NETCAT - nc

MFC after: 3 days
Sponsored by: Dell EMC Isilon

Conditionally compile certain programs into rescue(8) if requested

MK_CCD - ccdconfig
MK_ROUTED - routed, rtquery

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Add shutdown/poweroff support to rescue(8)

shutdown is a safer way to power off than reboot (in general), because of
the added shutdown process that it executes via /etc/rc.shutdown . It was
odd that it was missing from rescue(8) since reboot and friends were
added in past commits.

While here, alias poweroff to shutdown for parity with sbin/shutdown/Makefile

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes: yes

Split /rescue into its own package.

Sponsored by: The FreeBSD Foundation

Fix the build after ifconfig was converted over to lib80211 in r291470

Reported by: jenkins, O. Hartmann <ohartman@zedat.fu-berlin.de>
Pointyhat to: adrian

Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles(). It drops the
unused 'swapfile' argument and adds a new function pointer argument for
a symbol resolving function. Native kernels still use _fdnlist() from
libc to resolve symbols if a resolver function is not supplied, but cross
kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
Each backend is responsible for implementing kvm_read2() for a given
vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
powerpc cores).
- Add internal helper routines for the common page offset hash table used
by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
to be cross-friendly by using private constants instead of ones that
vary by platform (e.g. PAGE_SIZE). Static assertions are present when
a given backend is compiled natively to ensure the private constants
match the real ones.
- Enable all of the existing vmcore backends on all platforms. This means
that libkvm on any platform should be able to perform KVA translation
and read data from a vmcore of any platform.

Tested on: amd64, i386, sparc64 (marius)
Differential Revision: https://reviews.freebsd.org/D3341

Make iscsictl and iscsid build if MK_ISCSI == yes

MFC after: 1 month
X-MFC with: r289452

Add iscsictl(8) and iscsid(8) to rescue(8). The point is to make it
easier to build md_root images from rescue(8), to use with iSCSI boot.

The change increases the size of rescue by 62kB, from 8728kB to 8790kB.

Reviewed by: bapt@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3865

Removed extra line

Significant upgrades to sa(4) and mt(1).

The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.

o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.

o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.

This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.

For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.

o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.

o New, extensible tape driver parameter get/set interface.

o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.

o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.

o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.

o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.

o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.

o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.

This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.

rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.

src/share/man/man4/sa.4
Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.

Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.

Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.

Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.

Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.

Add a new MTIOCRBLIM ioctl to get block limits information.

Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().

scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.

Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.

For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.

Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.

Add a partition number to the sa(4) softc.

Improve device departure handling. The previous implementation
led to hangs when the device was open.

If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.

So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.

Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.

Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().

Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.

Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.

This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.

However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.

The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.

This includes definitions for the following new ioctls:

MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */

usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.

Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.

The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.

The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.

Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.

Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.

Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.

Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.

Sponsored by: Spectra Logic
MFC after: 1 month

Only build vi support into rescue if MK_VI != no

Sponsored by: EMC / Isilon Storage Division

Conditionally build ex, routed, rtquery, and vi support into rescue, based on
their respective MK_ flags

Add sleep(1) to /rescue. This adds 664 bytes to the binary on amd64 but
it's pretty useful for shell scripts.

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Add libxo, now needed by df(1).

Pointed out by: rodrigc@ (thanks!)

Remove duplicate prog.

Add zdb into rescue environment.

On amd64, this would increase the binary size by 1.1MiB and
make it possible to examine zpool status offline, useful for
recovery and diagnostic purposes.

Submitted by: sef
Obtained from: FreeNAS
MFC after: 2 weeks

In r232153, libarchive 3.0.3 was imported, replacing the archive_hash.h
header with archive_crypto_private.h, and its ARCHIVE_HASH_xxx macros
were renamed to ARCHIVE_CRYPTO_xxx.

Rename these macros in lib/libarchive/config_freebsd.h, to re-enable the
hashes for libarchive again. This affects the mtree format writer, and
the xar format reader and writer modules.

This also requires changes in the library order for statically linking
rescue, otherwise ld would complain about redefined symbols. Thanks to
jkim for pointing out the solution.

Reviewed by: kientzle
MFC after: 1 week

Replace all uses of libncurses and libtermcap with their wide character
variants. This allows usable file system images (i.e. those with both a
shell and an editor) to be created with only one copy of the curses library.

Exp-run: antoine
PR: 189842
Discussed with: bapt
Sponsored by: DARPA, AFRL

Remove ia64.

This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation

This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h

Discussed at: BSDcan

Minor style nits...

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

- Fix build WITHOUT_ZFS/WITHOUT_CDDL after r265229, bin/ps needs libjail.

MFC after: 2 weeks
X-MFC-with: r265229

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

Remove IPX support.

IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.

Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.

- Trim an unused and bogus Makefile for mount_smbfs.
- Reconnect with some minor modifications, in particular now selsocket()
internals are adapted to use sbintime units after recent'ish calloutng
switch.

Add less to rescue build. On amd64, this increases rescue size by about
130KB or 2.4%.

MFC after: 1 month

Remove all legacy ATA code parts, not used since options ATA_CAM enabled in
most kernels before FreeBSD 9.0. Remove such modules and respective kernel
options: atadisk, ataraid, atapicd, atapifd, atapist, atapicam. Remove the
atacontrol utility and some man pages. Remove useless now options ATA_CAM.

No objections: current@, stable@
MFC after: never

Disconnect non-MPSAFE SMBFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netsmb, which is a base
requirement for SMBFS.

In the while SMBFS regular users can use FUSE interface and smbnetfs
port to work with their SMBFS partitions.

Also, there are ongoing efforts by vendor to support in-kernel smbfs,
so there are good chances that it will get relinked once properly locked.

This is not targeted for MFC.

Disconnect non-MPSAFE NTFS from the build in preparation for dropping
GIANT from VFS. This code is particulary broken and fragile and other
in-kernel implementations around, found in other operating systems,
don't really seem clean and solid enough to be imported at all.
If someone wants to reconsider in-kernel NTFS implementation for
inclusion again, a fair effort for completely fixing and cleaning it
up is expected.

In the while NTFS regular users can use FUSE interface and ntfs-3g
port to work with their NTFS partitions.

This is not targeted for MFC.

Disconnect non-MPSAFE NWFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netncp, which is a base
requirement for NWFS.

In the possibility of a future maintenance of the code and later
readd to the FreeBSD base, maybe we should think about a better location
for netncp. I'm not entirely sure the / top location is actually right,
however I will let network people to comment on that more specifically.

This is not targeted for MFC.

unbreak build: rescue now needs -ljail if MK_ZFS is on

Disable jail support in ifconfig when either building a rescue
image or MK_JAIL knob has been set to "no".

Reviewed by: bz
Approved by: adrian (mentor)

-lreadline is not required anymore.

-lpthread is required by -lzfs so should be later in LIBS list.

There were no "undefined symbol pthread_xxx" errors during the link before
this fix only because of STATIC_LIB_REQUIRE() declarations in
lib/libthr/thread/thr_init.c.

Add netcat (nc) to /rescue.

MFC after: 3 weeks

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

Break out the rules which generate crunchgen'ed binaries into a separate
.mk file so they can be reused.

Introduce a new option, CRUNCH_BUILDTOOLS, which lists the binaries that
require tools built in the local architecture. sh and csh both require this.
It was previously hardcoded in rescue/rescue/Makefile .

Introduce a new option, CRUNCH_SHLIBS, which lists the shared libraries
to link against. These override the static libraries listed in CRUNCH_LIBS.
Some build environments may wish to use a handful of shared libraries
(eg libc.so) so other small, dynamic binaries can be run in the environment.

Remove the now-shared code from rescue/rescue/Makefile and introduce the
CRUNCH_BUILDTOOLS option for the above shells.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFtbemd:

Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.

Add xz to rescue.

Approved by: delphij (mentor)

Enable liblzma support in libarchive
Adjust dependencies for programs using libarchive
Add xz and linkage against liblzma to rescue system

Approved by: kientzle, delphij (mentor)
MFC after: 2 weeks

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Remove libulog from the bootstrap again.

libulog now only provides functions that are used by various packages
from the ports tree, namely the libutempter ones. There is no reason to
link it into the crunch/fixit binaries anymore.

Unbreak rescue(8). We should also link against libulog now.

Add the "vnet" and "-vnet" options, to allow moving interfaces between
jails with VIMAGE.

Approved by: bz (mentor)

Add zfs/zpool to rescue programs

PR: bin/125878
Submitted by: nork@
MFC after: 3 days

Libarchive is sprouting dependencies on libmd and libcrypto.
Because crunchgen drops any repeated library (keeping only the
first), the -lcrypto reference must be moved to after -larchive,
not merely duplicated.

I'm considering changing crunchgen's handling of duplicate
libraries, but that's a rather more delicate issue.

*** empty log message ***

Remove if_ppp(4) and if_sl(4).

Not only did these two drivers depend on IFF_NEEDSGIANT, they were
broken 7 months ago during the MPSAFE TTY import. if_ppp(4) has been
replaced by ppp(8). There is no replacement for if_sl(4).

If we see regressions in for example the ports tree, we should just use
__FreeBSD_version 800045 to check whether if_ppp(4) and if_sl(4) are
present. Version 800045 is used to denote the import of MPSAFE TTY.

Discussed with: rwatson, but also rwatson's IFF_NEEDSGIANT emails on the
lists.

Fix build when WITH_SSP is set explicitly.

Submitted by: Jeremie Le Hen

- Import the HEAD csup code which is the basis for the cvsmode work.

Take a moment to tidy some white space while I'm here. No functional
changes for this commit.

We don't need pax and tar. These days tar is a strict superset of
pax. Per discssuion on arch@ eliminate it.

pkill(1) first appeared in /usr/bin, but later it was moved to /bin
for the convenience of rc.d. Now it has happily lived there for quite
a while. So move the pkill(1) source files from usr.bin to bin, too.

Approved by: gad

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Remove netatm from HEAD as it is not MPSAFE and relies on the now removed
NET_NEEDS_GIANT. netatm has been disconnected from the build for ten
months in HEAD/RELENG_7. Specifics:

- netatm include files
- netatm command line management tools
- libatm
- ATM parts in rescue and sysinstall
- sample configuration files and documents
- kernel support as a module or in NOTES
- netgraph wrapper nodes for netatm
- ctags data for netatm.
- netatm-specific device drivers.

MFC after: 3 weeks
Reviewed by: bz
Discussed with: bms, bz, harti

Adding glabel alias killed gpart alias; fix it.

Add an alias for glabel(8).

o Build geom for all platforms.
o Don't build bsdlabel for ia64.
o Don't build fdisk and gpt for ia64.

Add some essential tools to rescue(8) in order to make it
a versatile emergency tool:

o sed(1) as a multi-purpose text filter -- can do grep's job and much more.
o head(1), tail(1), and tee(1) as idiomatic text filters.
o mt(1) to control tape drives (PR misc/98383).
o chown(8) aka chgrp(8) to complement the ch* subset.
o pkill(1) aka pgrep(1) to control running processes easily and thus to be
able to recover from a serious problem or a fatal typo in an otherwise live
system w/o a reboot. (It also deserves adding to rescue(8) for its having
triggered a latent bug in crunchgen(1), but we had better add a regression
test for that. :-)

The resulting change in rescue(8) size has the following order of magnitude
on i386: 3787656 - 3727872 = 59784, i.e. just a tad.

Discussed on: -hackers (I seem to have wearied all opponents :-)
PR: misc/98383

Disconnect netatm from the build as it is not MPSAFE and relies on
NET_NEEDS_GIANT, which will shortly be removed. This is done in a
away that it may be easily reattached to the build before 7.1 if
appropriate locking is added. Specifics:

- Don't install netatm include files
- Disconnect netatm command line management tools
- Don't build libatm
- Don't include ATM parts in rescue or sysinstall
- Don't install sample configuration files and documents
- Don't build kernel support as a module or in NOTES
- Don't build netgraph wrapper nodes for netatm

This removes the last remaining consumer of NET_NEEDS_GIANT.

Reviewed by: harti
Discussed with: bz, bms
Approved by: re (kensmith)

- Remove UMAP filesystem. It was disconnected from build three years ago,
and it is seriously broken.

Discussed on: freebsd-arch@
Approved by: re (mux)

Replace the GNU gzip with a slightly modified NetBSD gzip. The
NetBSD version is a feature-to-feature re-implementation of GNU
gzip using the freely-redistributable zlib and this version is
expected to be mostly bug-to-bug compatible with the GNU
implementation.

- Because this is a piece of mature code and we want to make
changes so it is added directly rather than importing to
src/contrib.
- Connect newly added code to src/usr.bin/ and rescue/rescue
build.
- Disconnect the GNU gzip code from build for now, they will
be eventually removed completely.
- Provide two new src.conf(5) knobs, WITHOUT_BZIP2_SUPPORT and
WITHOUT_BZIP2.

Tested by: kris (full exp-7 pointyhat build)
Approved by: core (importing a 4-clause BSD licensed file)
Approved by: re (adding new utility during -HEAD code slush)

Remove mount_ext2fs.

Remove references to mount_devfs, mount_fdescfs, mount_linprocfs, mount_procfs.

Reminded by: ru

Remove alpha left-overs.

Pass -DCRUNCH down to standard targets in individual makefiles.

*sigh*
Move the -lbsdxml after -lgeom, so that ld doesn't get confused and pretend
he can't find the symbol from libbsdxml needed in libgeom.
This should fix the rescue build breakage.

Extend coverage of the MK_IPX build option to the following:

- <netipx> headers [1]
- IPX library (libipx)
- IPX support in ifconfig(8)
- IPXrouted(8)
- new MK_NCP option

New MK_NCP build option controls:

- <netncp> and <fs/nwfs> headers
- NCP library (libncp)
- ncplist(1) and ncplogin(1)
- mount_nwfs(8)
- ncp and nwfs kernel modules

User knobs: WITHOUT_IPX, WITHOUT_IPX_SUPPORT, WITHOUT_NCP.

[1] <netsmb/netbios.h> unconditionally uses <netipx> headers
so they are still installed. This needs to be dealt with.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Install nextboot in /rescue as /rescue/nextboot rather than
/rescue/nextboot.sh to match the name in /sbin (/sbin/nextboot).

Reviewed by: gtetlow
MFC after: 1 week

Fix path to dhclient-script and reconnect to build.

Temporarily disconnect dhclient from the build while I import the
OpenBSD version.

Patches from Ruslam Ermilov to remove NetBSD bits from Makefiles and cleanup
build problems with rescue.

Run "make obj" before "make build-tools" in the directories used for
building the rescue binary. This fixes a problem with NO_TCSH, where
the "make obj" stage of buildworld doesn't recurse into bin/csh,
resulting in csh build-tools being put into /usr/src/bin/csh.

Pointed out by: dougb (on hackers@)

Wrap BSD r* commands in NO_RCMDS.
Change NO_RCMDNS to NO_RCMDS and do the switch in bsd.compat.mk.

Discussed with: ru, nectar

Add chroot. It can be a real lifesaver, and adds less than 2 kB.

Ignoring MAKEFLAGS in rev. 1.15 was a very bad idea. This causes
wrong share/mk files to be used, which can be fatal with upgrades
or downgrades, e.g., when building RELENG_5 on HEAD.

Reported by: glebius

For now, just exclude -P from MAKEFLAGS when running crunchgen(1).
(Note that it will still break when run with certain -d options.)

The real solution is to fix make(1) to not print stuff on stdout
when it's not supposed to, e.g., through the -P and -dX options,
and to fix crunchgen(1) to not redirect stderr to stdout when
running make(1). Once this is implemented, this hack can go.

Don't call "objs" target in rescue.mk twice.

Make "===> " prefixes look sane.

NOINET6 -> NO_INET6

NOCRYPT -> NO_CRYPT

NOATM -> NO_ATM

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Add missing dependencies of $(OUTPUTS) on source makefiles.

Caught by: make -DNOCLEAN buildworld

Replace GNU tar with BSD tar.

Remove references to NO_VINUM

For variables that are only checked with defined(), don't provide
any fake value.

As with the non-rescue version don't build fore_dnld when NOATM is defined.

Remove dangling raidctl reference

Add id(1) (aka groups(1) aka whoami(1)) since it is used by install.sh.

Revision 1.7 of this file added information
about the location of each program's source.

This update optimizes the build a bit by giving that
information to crunchgen rather than asking crunchgen
to do a directory search to locate sources.

Approved by: gordon (Mentor)

Properly spell make binary as ${MAKE}.

This fixes buildworld for systems with old make(1) binaries.

Reported by: Benjamin Close

Remove cxconfig and add sconfig

# maybe we can remove sconfig later from rescue?

Submitted by: Roman Kurakin <rik@cronyx.com>
Approved by: re@ <scottl>

Reconnect ipfstat, ipnat, and ipftest to the build now that if_xname
support is enabled.

Approved by: re (scottl)

Alternate version of rev 1.20.
Comment out rather than totally remove the ipfilter pieces that we need
reconnected some day. This is now only ipnat as it is for configuring NAT.
ipfstat is meant for reporting statistics/filter lists. For /rescue it is
enough to configure lists but not view the installed ones.

Temporarily disable ipfstat and ipnat in /rescue to fix world.

No need to check for the directory, fols are assumed to have all the
correct sources. Only check the crypto macros.

- Support for multibyte charsets in LIBICONV.
- CD9660_ICONV, NTFS_ICONV and MSDOSFS_ICONV kernel options
(with corresponding modules).
- kiconv(3) for loadable charset conversion tables support.

Submitted by: Ryuichiro Imura <imura@ryu16.org>

Teach rescue about NOATM, NO_VINUM and NOINET6

Build the atmconfig utility into rescue.

When using crunchgen, blow away MAKEFLAGS. This fixes make -j <#> -P
from passing the -P flag to crunchgen which seems to confuse crunchgen
horribly. This is the preferable solution to modifing crunchgen to
unset the MAKEFLAGS environment variable.

Submitted by: gad@

Fix rescue build using -j. The problem appears to be make not being able
to find the source when the object was specified as <directory>/foo.o.

The fix makes the build go through a make objs before compiling the rest
of the crunchgen. This ensures that the dhclient bits are built in the
correct place where they are picked up for the final compile of rescue.

I'd like to thank dwhite@ and gad@ for helping me track down the problem.

Fast testing box provided by: phk@ (thanks)

Missed one, remove one more.

Trim /rescue.

Discussed on: freebsd-arch@

Obey NOCRYPT

Submitted by: Skip Ford <skip.ford@verizon.net>

NO_TCSH would be set in /etc/make.conf, not here. Also obey NO_IPFILTER.

consistify the style some

Remove the commented out 'rmail' before someone gets a really bad idea.

Depessimize rescue build by only clean'ing and obj'ing the directories we
care about, not a full usr.bin tree. This should reduce buildworld times
pretty drastically.

Requested by: lots of people

Turn rescue back on, conditional to NORESCUE. We seem to be split on
using underscores or not, so I just randomly picked a style. I think
I have the logic correct, but if someone wants to give it a once over
that would be good.

Tim submitted a patch to fix the cross-building issues which I tested
with a tinderbox run for sparc64.

Submitted by: Tim Kientzle <kientzle@acm.org>

Remove mount_portalfs, it's just wrong in this context.

Don't build mount_nwfs or mount_smbfs in rescue.
Build fdisk_pc98 on pc98 arch, not fdisk.
Don't alias disklabel on pc98, ia64.
Don't build fdisk on sparc64, alpha.

Pointed out by: tmm@
Submitted by: Tim Kientzle <kientzle@acm.org>

Fix fdisk naming issues on pc98 and other platforms.
Fix disklabel, bsdlabel, and sunlabel on various platforms.

Noticed by: tmm@
Submitted by: Tim Kientzle <kientzle@acm.org>

Fix typo sparc -> sparc64

Submitted by: tmm@

Add /rescue bits. This basically encompasses all of bin and sbin along
with a couple of bits from usr.bin in a crunchgen'd binary.

Submitted by: Tim Kientzle <kientzle@acm.org>