bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

sh: accept fc options grouped behind one '-'

As per Utility Syntax Guidelines, accept both forms: -l -n and -ln.

To do that, anticipate the source string for the next option that will
be parsed by nextopt(). It's not always *argptr, sometimes it is
nextopt_optptr.

To simplify the check for not_fcnumber, slightly modify nextopt() to
always nullify nextopt_optptr in cases where it would have been set
to point to a NUL character.

Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D35836

sh: Add -o verify to use O_VERIFY when sourcing scripts

Add -o verify to sh to make it use O_VERIFY when
sourcing scripts and reading profiles.

Useful in conjunction with mac_veriexec to help protect at
least some parts of the boot sequence, e.g., /etc/rc*.

Differential revision: https://reviews.freebsd.org/D30464
Reviewed by: jilles, sjg
Obtained from: Stormshield

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

sh: Remove broken #ifdef NOHACK code (related to sh -c).

Apart from the fact that subtle syntactic changes make a poor compile-time
option, the NOHACK case has been obviously broken since it was added,
because it uses q uninitialized if (*p != '\0').

No functional change is intended.

sh: Update TOUR and comments for some code changes, some of them old.

Also, improve some terminology in TOUR and comments.

sh: Simplify setinteractive().

setsignal() does nothing if the signal disposition is already set correctly.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

sh: Avoid out-of-bounds access in setoptionbyindex() for 'set -o nolog'.

Reported by: hrs

sh: Simplify some code related to positional parameters.

sh: Reduce size of options table.

sh: Add set -o nolog.

POSIX requires this to prevent entering function definitions in history but
this implementation does nothing except retain the option's value. In ksh88,
function definitions were usually entered in the history file, even when
they came from ~/.profile and the $ENV file, to allow displaying their
definitions.

This is also the first option that does not have a letter.

sh: When setting option via long name, don't go via letter.

Looking up the letter makes no sense and prevents adding options that only
have a long name, no letter.

sh: Fix more compiler warnings.

sh: Make getopts memory-safe if with changing arguments.

POSIX does not permit to continuing a getopts loop with different
arguments. For parsing the positional parameters, we handle this case by
resetting the getopts state when the positional parameters are changed in
any way (and the getopts state is local to a function). However, in the
syntax getopts <optstring> <var> <arg...>, changes could lead to invalid
memory access.

In the syntax getopts <optstring> <var> <arg...>, store a copy of the
arguments and continue to use them until getopts is reset.

sh: In getopts, unset OPTARG where POSIX says we should.

sh: Don't discard getopts state on unknown option or missing argument.

When getopts finds an invalid option or a missing option-argument, it should
not reset its state and should set OPTIND as normal. This is an old ash bug
that was fixed long ago in dash. Our behaviour now matches most other
shells.

sh: Send getopts error messages to stderr, not stdout.

Adjust a testcase for this change.

sh: Add more necessary INTOFF/INTON.

sh: Remove XXX comment about removing nextopt().

Using nextopt() avoids depending on the BSD-specific optreset feature in
getopt() and reduces code size (both source and binary).

sh: Allow unsetting OPTIND.

Note that only assigning the decimal value 1 resets getopts, as before.

sh: Fix duplicate prototypes for builtins.

Have mkbuiltins write the prototypes for the *cmd functions to builtins.h
instead of builtins.c and include builtins.h in more .c files instead of
duplicating prototypes for *cmd functions in other headers.

sh: Improve error message if the script cannot be opened.

Avoid "<nosuchfile>: cannot open <nosuchfile>: ...".

sh: Check setuid()/setgid() return values.

If the -p option is turned off, privileges from a setuid or setgid binary
are dropped. Make sure to check if this succeeds. If it fails, this is an
error which will cause the shell to abort except in interactive mode or if
'command' was used to make 'set' or an outer 'eval' or '.' non-special.

Note that taking advantage of this feature and writing setuid shell scripts
seems unwise.

MFC after: 1 week

sh: Remove duplicate code resetting uid/gid for set +p/+o privileged.

MFC after: 1 week

sh: Remove special code for shell scripts without magic number.

These are called "shell procedures" in the source.

If execve() failed with [ENOEXEC], the shell would reinitialize itself
and execute the program as a script. This requires a fair amount of code
which is not frequently used (most scripts have a #! magic number).
Therefore just execute a new instance of sh (_PATH_BSHELL) to run the
script.

sh: Code size optimizations to buffered output.

This is mainly less use of the outc macro.

No functional change is intended, but code size is about 2K less on i386.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

In the spirit of r90111, depend on c89 and remove the "STATIC" macro
and its usage.

sh: Automatically enable -o emacs in interactive shells with terminals.

This makes sh a bit more friendly in single user mode, make buildenv, chroot
and the like, and matches other shells.

The -o emacs can be overridden on the command line or in the ENV file.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

sh: Various warning fixes (from WARNS=6 NO_WERROR=1):
- const
- initializations to silence -Wuninitialized (it was safe anyway)
- remove nested extern declarations
- rename "index" locals to "idx"

sh: Constify various strings.

Most of this is adding const keywords, but setvar() in var.c had to be
changed somewhat more.

- Import the HEAD csup code which is the basis for the cvsmode work.

Fix a bug in r177497 which caused the getopts state to be reset when 'set'
was used to set a shell option (and not to change the positional parameters).

Submitted by: Martin Kammerhofer

Reset the internal state used for the 'getopts' built-in when 'shift' or 'set'
are used to modify the arguments. Not doing so caused random memory reads or
null pointer dereferences when 'getopts' was called again later (SUSv3 says
getopts produces unspecified results in this case).

PR: 48318

Implement some of the differences between special built-ins and other builtins
demanded by POSIX.
- A redirection error is only fatal (meaning the execution of a shell script is
terminated) for special built-ins. Previously it was fatal for all shell
builtins, causing problems like the one reported in PR 88845.
- Variable assignments remain in effect for special built-ins.
- Option or operand errors are only fatal for special built-ins.
This change also makes errors from 'fc' non-fatal (I could not find any reasons
for this behaviour).

Somewhat independently from the above down-grade the error handling in the
shift built-in if the operand is bigger than $# from an error() call (which is
now fatal) to a return 1. I'm not sure if this should be considered a POSIX
"operand error", however this change is needed for now as we trigger that error
while building libncurses. Comparing with other shells, zsh does the same as
our sh before this change (write a diagnostic, return 1), bash behaves as our
sh after this commit (no diagnostic, return 1) and ksh93 and NetBSD's sh treat
it as a fatal error.

Include disabled options in the output of 'set +o'. POSIX says the output of
set +o can be used to reload previous settings, for this to work disabled
options must be printed as well or otherwise options that were set in the mean
time won't be turned off.

To avoid an excessively long output line I formatted the output to print only
six options per line.

Submitted by: Jilles Tjoelker
PR: 73500

A second attempt to adjust option-parsing on a shell command, for the
benefit of scripts start out as: #!/bin/sh -- # -*- perl -*-
With this fix in place, we can commit a change to kern/imgact_shell.c
so FreeBSD will process the `#!' line in shell-scripts in a more
standard fashion.

PR: 16393
Mentioned on: freebsd-arch

Change /bin/sh so *it* implements the processing needed for scripts to
work as expected when they have a "shebang line" of:

#!/bin/sh -- # -*- perl -*- -p

This specific line is recommended in some perl documentation, and I think
I've seen similar lines in documentation for ruby and python. Those
write-ups expect `sh' to ignore everything after the '--' if the first
thing after the '--' is a '#'. See chapter 19, "The Command-Line Interface"
in 3rd edition of "Programming Perl", for some discussion of why perl
recommends using this line in some circumstances.

The above line does work on solaris, irix and aix (as three data points),
and it used to work on FreeBSD by means of a similar patch to execve().
However, that change to execve() effected *all* shells (which caused
other problems), and that processing was recently removed.

PR: 16393 (the original request to fix the same issue)
Reviewed by: freebsd-current (looking at a slightly different patch)
MFC after: 1 week

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Consistently use FBSDID

Implement `set +o', which displays the currently set options in a format
suitable for re-input into the shell.

Usage style sweep: spell "usage" with a small 'u'.
Also change one case of blatant __progname abuse (several more remain)
This commit does not touch anything in src/{contrib,crypto,gnu}/.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.
o Change
int
foo() {
...
to
int
foo(void)
{
...

Fix warnings, some of them serious because sh violated name
spaces reserved by the header files it includes.

mkinit.c still produces C code with redundant declarations, although
they are more harmless since they automatically derived from the right
places.

$Id$ -> $FreeBSD$

Add rcsid. Spelling.

Avoid a possible NULL pointer dereference, nuke register usage,
and remove an unnecessary reset.

Obtained from: NetBSD

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Fix a problem caused by finger failure on my part. The builtin
getopts should now work as expected. This fix was in the NetBSD
code that I was merging from but missed getting into FreeBSD's
version because of 'drain bamage' on my part.

Submitted by: NetBSD, joerg

Merge in NetBSD mods and -Wall cleaning.

Obtained from: NetBSD, me

Add the -p (privileged) commandline switch
found in bash, zsh, and friends.

Reviewed by: joerg

Merge of 4.4-Lite2 sh source, plus some gcc -Wall cleaning. This is a
merge of parallel duplicate work by Steve Price and myself. :-]

There are some changes to the build that are my fault... mkinit.c was
trying (poorly) to duplicate some of the work that make(1) is designed to
do. The Makefile hackery is my fault too, the depend list was incomplete
because of some explicit OBJS+= entries, so mkdep wasn't picking up their
source file #includes.

This closes a pile of /bin/sh PR's, but not all of them..

Submitted by: Steve Price <steve@bonsai.hiwaay.net>, peter

Import the 4.4BSD-Lite2 /bin/sh sources

Requested by: joerg

(Note, this is mostly going to be conflicts, which is expected. Our entire
sh source has a mainline, so this should not change anything except for
a few new files appearing. I dont think they are a problem)

Fix my breakage of the $0 handling during $ENV processing.

Since the broken version went into 2.1, this fix should, too.

Posixize:

sh -c [-aCefinuvx] command_string [ command_name [argument ...] ] 1

4.56.3 Options

-c Read commands from the command_string operand. Set the
value of special parameter 0 (see 3.5.2) from the value of
the command_name operand and the positional parameters
($1, $2, etc.) in sequence from the remaining argument
operands.

Pointed out by: Kaleb Keithly (kaleb@x.org)

Fix /bin/sh's broken handling of the builtin getopts(1). The options
``-ffoo'' and ``-f foo'' have been treated differently.

This has been in violation of Posix.2 (that deprecates -ffoo, but
doesn't disallow it).

Remove trailing whitespace.

Reviewed by: phk

Added $Id$

BSD 4.4 Lite bin Sources