bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

sh: Don't allocate a redirtab if there are no redirections.

Builtins (including variable assignments without command word), function
calls and redirected compound commands need to restore file descriptors
to their original state after execution. This is handled by allocating a
redirtab structure. These mallocs and frees show up heavily in pmcstat.

Only allocate a redirtab if there are actually redirections and maintain a
count of how many levels of REDIR_PUSH there are without redirtabs.

A simple loop without external programs like

sh -c 'i=0; w=$(printf %0100d 7); while [ "$i" -lt 1000000 ]; do
i=$((i+1)); done'

is over 25% faster on an amd64 bhyve VM.

sh: Eliminate some gotos.

sh: Fix possible memory leaks and double frees with unexpected SIGINT.

sh: Add some consts.

sh: Make <&0 disable the </dev/null implicit in a background command.

Although <&0 does nothing, it is a redirection affecting standard input and
should therefore disable the </dev/null redirection implicit in a background
command.

sh: Remove unnecessary reset functions.

These are already handled by exception handlers.

sh: Remove #define MKINIT.

MKINIT only served for the removed mkinit. Many variables can be static now.

sh: Remove mkinit.

Replace the RESET blocks with regular functions and a reset() function that
calls them all.

This code generation tool is unusual and does not appear to provide much
benefit. I do not think isolating the knowledge about which modules need to
be reset is worth an almost 500-line build tool and wider scope for
variables used by the reset functions.

Also, relying on reset functions is often wrong: the cleanup should be done
in exception handlers so that no stale state remains after 'command eval'
and the like.

sh: Use O_CLOEXEC and F_DUPFD_CLOEXEC instead of separate fcntl() call.

sh: Write as much into the heredoc pipe as possible, to avoid forking.

Use non-blocking I/O to write as much as the pipe will accept (often 64K,
but it can be as little as 4K), avoiding the need for the ugly PIPESIZE
constant. If PIPESIZE was set too high, a deadlock would occur.

sh: Expand here documents in the current process.

Expand here documents at the same point other redirections are expanded but
use a non-fork subshell environment (like simple command substitutions) for
compatibility. Substitition errors result in an empty here document like
before.

As a result, a fork is avoided for short (<4K) expanded here documents.

Unexpanded here documents (with quoted end marker after <<) are not affected
by this change. They already only forked when >4K.

Side effects:
* Order of expansion is slightly different.
* Slow expansions are not executed in parallel with the redirected command.
* A non-fork subshell environment is subtly different from a forked process.

sh: Remove special code for shell scripts without magic number.

These are called "shell procedures" in the source.

If execve() failed with [ENOEXEC], the shell would reinitialize itself
and execute the program as a script. This requires a fair amount of code
which is not frequently used (most scripts have a #! magic number).
Therefore just execute a new instance of sh (_PATH_BSHELL) to run the
script.

sh: Check if dup2 for redirection from/to a file succeeds.

A failure (e.g. caused by ulimit -n being set very low) is a redirection
error.

Example:
ulimit -n 9; exec 9<.

sh: Check whether dup2 was successful for >&FD and <&FD.

A failure (usually caused by FD not being open) is a redirection error.

Exp-run done by: pav (with some other sh(1) changes)

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

In the spirit of r90111, depend on c89 and remove the "STATIC" macro
and its usage.

Consistently use "STATIC" for all functions in order to be able to set
breakpoints with in a debugger. And use naked "static" for variables.

Noticed by: bde

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Fix some cases where file descriptors from redirections leak to programs.

- Redirecting fds that were not open before kept two copies of the
redirected file.
sh -c '{ :; } 7>/dev/null; fstat -p $$; true'
(both fd 7 and 10 remained open)
- File descriptors used to restore things after redirection were not
set close-on-exec, instead they were explicitly closed before executing
a program normally and before executing a shell procedure. The latter
must remain but the former is replaced by close-on-exec.
sh -c 'exec 7</; { exec fstat -p $$; } 7>/dev/null; true'
(fd 10 remained open)

The examples above are simpler than the testsuite because I do not want to
use fstat or procstat in the testsuite.

Fix various things about SIGINT handling:
* exception handlers are now run with interrupts disabled, which avoids
many race conditions
* fix some cases where SIGINT only aborts one command and continues the
script, in particular if a SIGINT causes an EINTR error which trumped the
interrupt.

Example:
sh -c 'echo < /some/fifo; echo This should not be printed'
The fifo should not have writers. When pressing ctrl+c to abort the open,
the shell used to continue with the next command.

Example:
sh -c '/bin/echo < /some/fifo; echo This should not be printed'
Similar. Note, however, that this particular case did not and does not work
in interactive mode with job control enabled.

Fix race condition in noclobber option.

Formerly, it was possible for the file to be created between the check if it
existed and the open; the contents would then be lost.

Because this must use O_EXCL, noclobber > will not create a file through a
symlink anymore. This agrees with behaviour of other shells.

Approved by: ed (mentor) (implicit)

- Import the HEAD csup code which is the basis for the cvsmode work.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Replace home-grown dup2() implementation with actual dup2() calls. This
should slightly reduce the number of system calls in critical portions of
the shell, and select a more efficient path through the fdalloc code.

Reviewed by: bde

Changes following CScout analysis:

- Removed dead declarations
- Made objects that should have been declared as static, static.

The changes use STATIC instead of static, following the existing
convention in the rest of the code.

Approved by: schweikh (mentor)
MFC after: 2 weeks

Remove dead code which supported systems without O_APPEND, O_CREAT or SIGTSTP.

Convert the remaining callers of errmsg() to use strerror(), and remove
errmsg() and its table of error messages.

Don't assume file descriptors fit in a short, use an int instead.

Close file descriptors when [n]>&- and [n]<&- redirections are used.
This was broken by rev. 1.16.

PR: 40334
MFC after: 1 week

Consistently use FBSDID

Implement the -C (-o noclobber) option, which prevents existing regular
files from being overwritten by shell redirection.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.
o Change
int
foo() {
...
to
int
foo(void)
{
...

The fix for >/dev/stdout, including Tor Egge's fix for the bug in the
original attempt of the fix. And yes, this time I've tried to build
world with it and it succeeded.

Submitted by: Tor Egge
MFC after: 1 week

backout rev 1.14, it's breaking things.

Added ability to do ">/dev/stdout". The apsfilter people are anxious
to get it MFCed in time for release 4.5.

MFC after: 2 weeks

Implement the <> redirection operator.

Include strerror(errno) in error messages after failed system calls.
Fix a warning.

$Id$ -> $FreeBSD$

Add rcsid. Spelling.

Fix redirection of unopened file descriptors and nuke register
keyword usage.

Obtained from: NetBSD

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Merge in NetBSD mods and -Wall cleaning.

Obtained from: NetBSD, me

Merge of 4.4-Lite2 sh source, plus some gcc -Wall cleaning. This is a
merge of parallel duplicate work by Steve Price and myself. :-]

There are some changes to the build that are my fault... mkinit.c was
trying (poorly) to duplicate some of the work that make(1) is designed to
do. The Makefile hackery is my fault too, the depend list was incomplete
because of some explicit OBJS+= entries, so mkdep wasn't picking up their
source file #includes.

This closes a pile of /bin/sh PR's, but not all of them..

Submitted by: Steve Price <steve@bonsai.hiwaay.net>, peter

Import the 4.4BSD-Lite2 /bin/sh sources

Requested by: joerg

(Note, this is mostly going to be conflicts, which is expected. Our entire
sh source has a mainline, so this should not change anything except for
a few new files appearing. I dont think they are a problem)

o rename ulimit -p into ulimit -u, so we are in agreement with bash

o fix brokeness for 1>&5 redirection, where `5' was an invalid file
descriptor, but no error message has been generated

o fix brokeness for redirect to/from myself case

Remove trailing whitespace.

Reviewed by: phk

Added $Id$

BSD 4.4 Lite bin Sources