Reduce reliance on sys/sysproto.h pollution

Add sys/errno.h, sys/malloc.h, sys/queue.h, and vm/uma.h as needed.

sys/sysproto.h currently includes sys/acl.h which currently includes
sys/param.h, sys/queue.h, and vm/uma.h which in turn bring in
sys/errno.h sys/malloc.h.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D44465

sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

sbin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

restore: fix restore of NFS4 ACLs

Changing the mode bits on a file with an NFS4 ACL results in the
NFS4 ACL being replaced by one matching the new mode bits being set,
so when restoring a file with an NFS4 ACL, set the owner/group/mode first
and then set the NFS4 ACL, so that setting the mode does not throw away
the ACL that we just set.

Reviewed by: mckusick
Differential Revision: https://reviews.freebsd.org/D37618

Clean up global variable declarations in the dump and restore
utilities so that they will compile with -fno-common.

Started by: Kyle Evans (kevans)
Reviewed by: Kyle Evans (kevans)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24210

Re-enable reading byte swapped NFS_MAGIC dumps.

Fix bug introduced in r98542: previously to this revision the byte-swapped
value was compared at this place. The current check is in a conditional
section where the non-byte-swapped value was already checked to be not
the value which is checked again. As byte-swapping is activated afterwards,
it only makes sense if the byte-swapped value is checked.

Submitted by: Keith White <kwhite@site.uottawa.ca>
PR: 200059
MFC after: 1 month
Sponsored by: Essen Hackathon

Revert r313780 (UFS_ prefix)

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9623

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

prefix UFS symbols with UFS_ to reduce namespace pollution

Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Also prefix ext2's and nandfs's NDADDR and NIADDR with EXT2_ and NANDFS_

Reviewed by: kib, mckusick
Obtained from: NetBSD
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9536

restore(8): Handle extended attribute names correctly

UFS2 extended attribute names are not NUL-terminated. Handle
appropriately.

Correct the EXTATTR_BASE_LENGTH() macro, which handled ea_namelength ==
one (mod eight) extended attributes incorrectly.

PR: 216127
Reported by: dewayne at heuristicsystems.com.au
Reviewed by: kib@
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9208

restore: promote some getfiles() parameters to size_t.

This is based on a change from OpenBSD:

"Fix restore so that it can actually restore files larger than 4GB by
changing the type of "size" to off_t in getfiles() plus little dependent
type cleanup, from Daniel Lucq."

It is an important for machines with 32 bit longs.
While here unsign the flags, also from OpenBSD.

Obtained from: OpenBSD (through bitrig, I hate CVS)
MFC after: 2 weeks

Use MIN()/MAX() macros from sys/param.h.

MFC after: 2 weeks.

restore(8): fix use of uninitialized value.

Prevent uninitialized use of scalar newvol when jumping to gethdr.

CID: 1006491

If you backup a large file that is mostly holes, previously we'd issue
a seek for every block... For large (Exabyte sized files) this would
issue lots of unneeded seeks, so combine them...

Thanks for the work Jan, sorry took so long to commit... And an item
completed off the IdeasPage!

Submitted by: Jan Sucan

restore: Preserve timestamps to the nanosecond.

The restore utility already knows the full-resolution timestamps, so the
only thing to do is to stop converting the timespecs to timevals and use
futimens() and utimensat().

Differential Revision: https://reviews.freebsd.org/D2020
Reviewed by: jhb

When restoring a UFS dump onto a ZFS filesystem, an assertion in
restore was failing because ZFS was reporting a blocksize that was
not a multiple of 1024. Replace restore's failed assertion with
code that writes restored files in a blocksize that works for
restore (a multiple of 1024) despite being non-optimal for ZFS.

Submitted by: Dmitry Morozovsky
Tested by: Dmitry Morozovsky
MFC after: 1 week

Check the return error of set[e][ug]id. While this can never fail in the
current version of FreeBSD, this isn't guarenteed by the API.
Custom security modules, or future implementations of the setuid and
setgid may fail.

Submitted by: Erik Cederstrand
Approved by: cperciva
MFC after: 3 days

Fix sbin/ build with a 64-bit ino_t.

Original code by: Gleb Kurtsou

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

MFC r203157, r203816:

Handle short reads when the -P option is used and remove some dead code.

PR: bin/121502

Don't try to determine tape block size when the -P option is used.
This was missed in r203157.

PR: bin/121502

- Handle short reads when the -P option is used. Short reads must be
handled when reading from pipes.
- Remove dead code related to the -P option from getvol(). pipein and
pipecmdin are never set at the same time.

PR: bin/121502
Approved by: trasz (mentor)
MFC after: 2 weeks

- Cast time_t, int64_t and some int32_t values to intmax_t and use "%jd"
in format strings.
- Use (void) instead of (void *) when discarding strcat(3) return value.
- Format string fixes to match variable types.
- Change canon() len parameter and getcmd() size parameter type from
int to size_t.
- Style Makefile and increase WARNS to 2.

PR: bin/140061
Submitted by: uqs
Approved by: trasz (mentor)

- Import the HEAD csup code which is the basis for the cvsmode work.

This fixes the "getfile: lost data" panic when restoring dumps
on a 7.0 or later system that were created on a pre-5.0 system.
We must ensure that restore zeros out the previously undefined
birthtime and external attribute size fields when reading dump
tapes made by the UFS1 dump program.

The problem is that UFS2 dump carefully zeros out the unused
birthtime and external attribute size fields in the dump header
when dumping UFS1 filesystems, but the UFS1 dump didn't know about
those fields (they were spares) so just left whatever random junk
was in them. So, when restoring one of these pre-UFS2 dumps,
the new restore would eventually trip across a header that had
a non-zero external attribute size and try to extract it. That
consumed several tape blocks which left it totally out of sync
and very unhappy (i.e., the panic). The fix is in the gethead()
function which modernizes old headers by copying old fields to
their new location (and with this fix) zeroing out previously
undefined fields.

PR: bin/120881
Review by: David Malone & Scott Lambert
MFC after: 1 week

Follow on to fix 1.51 for "Header with wrong dumpdate" message.

Must ensure that dump tapes from UFS1 filesystems properly copy
old fields of dump headers to new locations. Move check of dumpdate
to follow the code which ensures that the appropriate fields have
been copied.

PR: bin/118087
Help from: David Malone, Scott Lambert, Javier Martín Rueda
MFC after: 2 weeks

Avoid printing spurious ``Header with wrong dumpdate.'' message.

Correctly set file group when restore is run by a user other than root.

Move macros describing extended attributes in UFS from
<sys/extattr.h> to <ufs/ufs/extattr.h>. Move description
of extended attributes in UFS from man9/extattr.9 to
man5/fs.5.

Note that restore will not compile until <sys/extattr.h>
and <ufs/ufs/extattr.h> have been updated.

Suggested by: Robert Watson

Update the dump program to save extended attributes. Update
the restore program to restore all dumped extended attributes.

If the restore is running as root, it will always be able
to restore all extended attributes. If it is not running
as root, it makes a best effort to set them. Using the -v
command line flag or the `verbose' command in interactive
mode will display all the extended attributes being set on
files (and at the end on directories) that are being restored.
It will note any extended attributes that could not be set.

The extended attributes are placed on the dump image immediately
following each file's data. Older versions of restore can work
with the newer dump images. Old versions of restore will
correctly restore the file data and then (silently) skip
over the extended attribute data and proceed to the next file.

This resolves PR 93085 which will be closed once the code
has been MFC'ed.

Note that this code will not compile until these header
files have been updated: <protocols/dumprestore.h> and
<sys/extattr.h>.

PR: bin/93085
Comments from: Poul-Henning Kamp and Robert Watson
MFC after: 3 weeks

Add a "-D" flag to restore which puts it into "degraded" mode. This
makes restore less efficient, but it makes a bigger effore to read
corrupted dumps. Specifiacally, when in degreded mode:

1) Restore shifts the input by 1 byte if it sees a problem,
rather than one tape block.
2) It doesn't assume the inodes are stored in ascending order.
3) It turns some panics into warning printfs.

We also verify some fields more carefully than before.

There's probably more a degreded mode could do, but this seems to
help a lot.

Approved by: imp, iedowse, mckusick
MFC after: 3 weeks

o Fix style(9) for previous.

A bunch of fixes from NetBSD:

o Restore owner/group/mode/atime/mtime of symbolic links, rev. 1.30.
o Extract file flags of symbolic link, rev. 1.42.
o Call getfile() before altering file attributes.
Open file with mode 0600 instead of 0666 so that file won't remain
group or world readable/writable even if getfile() terminated.
Move skipfile() before altering file attributes in IF{CHR,BLK} and
IFIFO case for symmetry, rev. 1.32.
o Use file mode 0600 when creating special file or fifo, revs. 1.33, 1.34.

o Remove redundant -N check.

PR: bin/101660
Submitted by: Andrey V. Elsukov
Obtained from: NetBSD, enami@netbsd
MFC after: 6 weeks

rscid -> __FBSDID. Mark parameter as __unused when necessary.

Restore the ability to read FreeBSD 1 tapes (and I think any net2
based tapes, but I'm not sure where NFS_MAGIC was introduced after
4.3). When support for the pre-4.4 format was removed (the ability to
read 4.2 and 4.3 BSD tapes), the old format inode conversion was
junked as well. However, FreeBSD 1 dump tapes use the NFS_MAGIC
format, but have this inode format. Before, restore would fail
complaining that '.' wasn't found and the root directory wasn't on
this tape. Since the conversion from the not so old format is
relatively trivial, restore the code to make that conversion.

FreeBSD 1 dumps are once again readable.

MFC After: a few days

c_tapea and c_firstrec are used for TS_TAPE blocks, so convert them
for the old (4.4-lite through FreeBSD 4.x and *BSD) format. It looks
like they aren't used for TS_INODE, but conversion costs so little
there that I've not removed them there (in case my grep was wrong).

This makes at least some of the tapes work for me again. Now, to
regresion test all my dusty tapes...

In order to print out the dump dates correctly, the date and ddate fields
also need to be convereted for old tapes for records of type TAPE.

Include <timeconv.h> for time conversion functions.

Approved by: das (mentor)

Add -P arguments for dump(8) and restore(8) which allow the user to
use backup methods other than files and tapes. The -P argument is
a normal sh(1) pipeline with either $DUMP_VOLUME or $RESTORE_VOLUME
defined in the environment, respectively.

For example, I can back up my home to three DVD+R[W]s as so:
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/ad0s2e 40028550 10093140 26733126 27% /home
green# dump -0 -L -C16 -B4589840 -P 'growisofs -Z /dev/cd0=/dev/fd/0' /home

Remove advertising clause from University of California Regent's license,
per letter dated July 22, 1999.

Approved by: core, imp

Use the standardized CHAR_BIT constant instead of NBBY in userland.

Change utimes to set the file creation time (for filesystems that
support creation times such as UFS2) to the value of the
modification time if the value of the modification time is older
than the current creation time. See utimes(2) for further details.

Sponsored by: DARPA & NAI Labs.

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

more file system > filesystem

Address a few minor style and consistency issues in revision 1.32.

Submitted by: Joshua Goodall <joshua@roughtrade.net>

Set the permissions on restored symbolic links.

PR: bin/37665
Submitted by: "Michael C. Adler" <mad1@tapil.com>

o remove __P
o Use ANSI function definitions
o unifdef -D__STDC__

Remove 'register' keyword.
It does not help modern compilers, and some may take some hit from it.
(I also found several functions that listed *every* of its 10 local vars with
"register" -- just how many free registers do people think machines have?)

Use a more robust scheme for determining how many blocks to skip
after an EOT-terminated volume. We keep track of the current record
number, and synchronise it with the c_tapea field each time we read
a header. Avoid the use of c_firstrec because some bugs in dump can
cause it to be set incorrectly.

Move the initialisation of some variables to avoid compiler warnings.

When we reach the end of the dump in findinode(), ask for another
volume if we missed some earlier tapes (the user can still enter
'none' later if the tapes are unavailable). Previously with 'x'
restores, we might not ask for all tapes if the tapes are supplied
in reverse order.

Clarify the message that describes what volume should be mounted
first; reverse order is only efficient when extracting a few files.

Fix a number of long-standing restore bugs in tape.c, mainly relating
to multi-volume restores:
- In findinode(), keep a copy of header->c_type so that we don't
exit the do-while loop until we have processed the current header.
Exiting too early leaves curfile.ino set to 0, which confuses
the logic in createfiles(), so multi-volume restores with the
'x' command don't work if you follow the instructions and supply
the tapes in reverse order. This appears to have been broken
by CSRG revision 5.33 tape.c (Oct 1992).
- The logic in getvol() for deciding how many records to skip after
the volume header was confused; sometimes it would skip too few
records and sometimes too many, leading to "resync restore"
warnings and missing files. Skip to the next header only when
the current action is not `USING'. Work around a dump bug that
sets c_count incorrectly in the volume header of the first tape.
Some of the problems here date back to at least 1991.
- Back out revision 1.23. This appeared to avoid warnings about
missing files in the 'rN' verification case, but it made the
problems with the 'x' command worse by stopping getvol() from
even attempting to find the first inode number on the newly
inserted tape. The bug it addressed is fixed by correcting the
skipping logic as described above.
- Save the value of `tpblksread' in case the wrong volume is
supplied, because it is incremented each time we read a volume
header. We already saved `blksread' for the same reson.

Don't refer to findinode()'s `complain' parameter in a comment; it
was removed in 1986.

I've been meaning to do this for a while. Add an underscore to the
time_to_xxx() and xxx_to_time() functions. e.g. _time_to_xxx()
instead of time_to_xxx(), to make it more obvious that these are
stopgap functions & placemarkers and not meant to create a defacto
standard. They will eventually be replaced when a real standard
comes out of committee.

Make the protocol/dumprestore.h header match restore's idea of the dump
header for the case where sizeof(time_t) != sizeof(int). dumprestore.h
was embedding time_t when it should have been embedding int32_t.

Use time_to_time32() and time32_to_time() to convert between the
protocoll/file-format time and time_t.

Add 'SKIP' as an action so that verification works for multivolume restores.
Tested with filesystem files.

PR: 27218
Submitted by: mad1@tapil.com
MFC after: 3 weeks

Move _PATH_DEFTAPE to <paths.h> to remove all the duplication of definitons,
and remove leading `r'(aw) from it.

Stop restore from looping under certain error conditions. This
corrects cases where restore would spew an infinite stream of
"Changing volumes on pipe input?" messages, or would loop waiting
for a response to the "set owner/mode for '.'" question.

PR: bin/14250
Reviewed by: dwmalone

Don't give up on file write errors. Just log them and continue.

Don't depend on <sys/stat.h> bogusly including <sys/time.h> (and thereby
<time.h>).

Fix sign extension.

PR: 21232
Obtained from: Christian Weisgerber <naddy@mips.inka.de>

Don't try to make files immutable (ie. chflags) before setting access times.

PR: 19973
Submitted by: Arjan de Vet <Arjan.deVet@adv.iae.nl>
Reviewed by: Matthew Jacob <mjacob@feral.com>

$Id$ -> $FreeBSD$

64bit fixes.
(Note: ``dump'' doesn't work on alpha yet. Apparently there's a problem
somewhere is the physio() area)

Submitted by: myself && Matt Dillon.

Commit patch from Tor Egge to fix the "large filesystem restore" problem.
This appears to work for me in the old case, but I don't have large
enough filesystems to test the fix case.

Reported working by: karl@mcs.net

Add rcsid. Remove unused #includes. Add missing prototypes and others -Wall
cleanings. Spelling.

Fixed printf format errors. Sure is dusty here.

Well, nobody objected, so here's my -u (unlink) flag to restore.

The logic in tape.c:getfile() doesn't allow for a filesystem
with a blocksize smaller than the tape block size. The problem
seems to be most easily fixed by changeing where fssize is set.

PR: 5704
Submitted by: David Malone <dwmalone@maths.tcd.ie>

Typo fix in the message displayed.

Merge from Lite2
- cleanups,
- whiteout support
- bug fixes (chflags missing on a few file types etc)
The dump/restore folks would want to have a closer look at this, the
change is pretty big.

Import some CSRG 4.4BSD-Lite2 components for sbin onto vendor branch.
(note that some of these have already been imported, this is a no-op)

Fix a minor problem with restore from tapes from big endian machines.
This finishes the closing of PR2446 which J"org Wunsch pointed out to
me after I closed it.

Submitted by: Tor Egge <Tor.Egge@idt.ntnu.no>

Yet another buffer overflow.

2.2 candidate
(and -stable too actually, who does that?)
Reviewed by: Warner Losh

Various security related deltas from OpenBSD

dirs.c:
From OpenBSD 1.2, 1.3, 1.5, 1.8, 1.10, 1.11, 1.12
1.2:
use unique temporary files; netbsd pr#2544;
lukem@supp.cpr.itg.telecom.com.au
1.3:
updated patch from lukem@supp.cpr.itg.telecom.com.au
to also make -r and -R work again
1.5:
mktemp open & fdopen
1.8:
/tmp// -> /tmp/
1.10:
Fix strncpy usage and correct strncat length field,
from Theo. Also change some occurrence of MAXPATHLEN
with sizeof(foo).
1.11:
does noone know how to use strncat correctly?
1.12:
use mkstemp()
From NetBSD:
Use open rather than create so we can specify
exclusive open mode.

main.c:
From OpenBSD 1.2, 1.5
1.2:
From NetBSD: support $TAPE.
1.5
Set umask to be read only by owner until we set real
file permissions.
tape.c:
From NetBSD:
Use open rather than create so we can specify
exclusive open mode.

ts_sec -> tv_sec
ts_nsec -> tv_nsec

Remove trailing whitespace.

Added FIFO restore capability.

BSD 4.4 Lite sbin Sources

Note: XNSrouted and routed NOT imported here, they shall be imported with
usr.sbin.