bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

sh: also auto-complete functions

Differential Revision: https://reviews.freebsd.org/D40619

sh: tab-complete aliases

Differential Revision: https://reviews.freebsd.org/D40619

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

sh: also auto-complete built-ins

Reported in a comment in PR 261728.

Reported by: Oleg
Reviewed by: jilles (previous version), bapt
Differential Revision: https://reviews.freebsd.org/D39839

sh: improve style

sh(1): fix history file write checking

We cannot just compare histsizeval() against 0, since that returns
a string pointer, which is always non-zero (non-null). The logic
in sethistsize() initializes the history size to 100 with values
that are non-number, and an empty string counts as that. Therefore,
the only time we want to not write into history with HISTSIZE val
set is when it's explicitly 0.

MFC after: 2 weeks

sh(1): Allow non-printing characters in prompt strings

Introduce new prompt format characters:

- '\[' starts the sequence of non-printing chatacters
- '\]' ends the sequence of non-printing characters

Within these sequences, the following characters are now supported:

- '\a' emits ASCII BEL (0x07, 007) character
- '\e' emits ASCII ESC (0x1b, 033) character
- '\r' emits ASCII CR (0x0d, 015) character
- '\n' emits ASCII CRLF sequence

These can be used to embed ANSI sequences into prompt strings.

Example in .shrc:

PS1="\[\e[7m\]\u@\h\[\e[0m\]:\w \\$ "

This tries to maintain some degree of compatibility with GNU bash,
that uses GNU readline library (which behaves slightly different from
BSD editline): It has two "non-printing boundary" characters:

- RL_PROMPT_START_IGNORE (\001)
- RL_PROMPT_END_IGNORE (\002)

while BSD editline only has one (when using EL_PROMPT_ESC setting), so
for this purpose, ASCII \001 was chosen and both \[ and \] emits
this character.

And while here, enlarge PROMPTLEN from 128 to 192 characters.

Reviewed by: jilles
Approved by: jilles
Differential Revision: https://reviews.freebsd.org/D37701

sh: accept fc options grouped behind one '-'

As per Utility Syntax Guidelines, accept both forms: -l -n and -ln.

To do that, anticipate the source string for the next option that will
be parsed by nextopt(). It's not always *argptr, sometimes it is
nextopt_optptr.

To simplify the check for not_fcnumber, slightly modify nextopt() to
always nullify nextopt_optptr in cases where it would have been set
to point to a NUL character.

Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D35836

sh: fix autocompletion for commands that share name with a directory

Provide libedit a special function making it always add a space after
the autocompleted command. The default one adds a slash if the word is
also a name of a directory in the current working directory, but this is
wrong for commands.

Reviewed by: bapt, jilles
Differential Revision: https://reviews.freebsd.org/D34544

sh: Add -o verify to use O_VERIFY when sourcing scripts

Add -o verify to sh to make it use O_VERIFY when
sourcing scripts and reading profiles.

Useful in conjunction with mac_veriexec to help protect at
least some parts of the boot sequence, e.g., /etc/rc*.

Differential revision: https://reviews.freebsd.org/D30464
Reviewed by: jilles, sjg
Obtained from: Stormshield

sh: try to avoid overwriting HISTFILE produced by other shells

If an attempt to load history from an existing history file was
unsuccessful, do not try to save command history to that file on exit.

sh: improve command completion

When there are many matches, find the longest common substring starting
from the beginning of each command and use that to replace input.

As an example: on my system, llv<tab> will be autocompleted to llvm-
and another <tab> will print all matching llvm commands.

sh: reset sh bindings on bind -e, bind -v

Until this change, any bindings set in histedit() were lost on calls to
bindcmd().

Only bind -e and bind -v call libedit's keymacro_reset(). Currently you
cannot fool libedit/map.c:map_bind() by trying something like bind -le
as when p[0] == '-', it does a switch statement on p[1].

sh: remove emacs improvements that are now defaults in libedit

sh: improve command completion

When multiple matches are found, we keep the provided string on the
input line and print unique matches as suggestions.

But the multiple matches might be the same command found in different
directories, so we should deduplicate the matches first and then decide
whether to autocomplete the command or not, based on the number of
unique matches.

sh: improve emacs mode

in emacs mode ^W should delete the previous word by default
Note that upstreaming this change directly into libedit is in process.

Reported by: manu
Reviewed by: jills, pstef, manu
Differential Revision: https://reviews.freebsd.org/D29493

sh: fix NO_HISTORY build

Move code added in b315a7296d2a ("autocomplete commands") to
conditionally compiled part under #ifndef NO_HISTORY.

Reported by: bdrewery
Fixes: b315a7296d2a

sh: implement persistent history storage

Implement persistent history storage:
the strategy is simple at start: loads the existing .sh_history file
at exit dump it.

The implementation respects the HISTFILE variable and its POSIX
definition: ~/.sh_history is used if HISTFILE is not set.

to avoid sh to create the history file, set HISTSIZE to 0 or HISTFILE to
en empty value

Co-authored-by: pstef
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D29493

sh: improve emacs mode

In emacs mode, force ^R to backware search the history
This behaviour is the default in emacs mode for most of the other shells

Note: Note that this can still be overridden via $EDITRC, ~/.editrc or a
bind command after set -o emacs.

MFC after: 1 week
Approved by: jilles
Reviewed by: jilles, arichardson, pstef
Differential Revision: https://reviews.freebsd.org/D29494

sh(1): autocomplete commands

Without this patch, sh can autocomplete file names but not commands from
$PATH. Use libedit's facility to execute custom function for autocomplete,
but yield to the library's standard autocomplete function when cursor is
not at position 0.

Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D29361

Fix build with NO_HISTORY set

Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D24458

Do not use our custom completion function, it is not needed anymore

sh: Send normal output from bind builtin to stdout

PR: 233343
Submitted by: Yuichiro NAITO (original version)

sh: Send libedit "ferr" output to fd 2

The libedit "fout" output must be sent to fd 2 since it contains prompts
that POSIX says must be sent to fd 2. However, the libedit "ferr" output
receives error messages such as from "bind" that make no sense to send to fd
1.

sh: Call fc -e editor with interrupts enabled.

Starting the fc -e editor can execute arbitrary script, and executing
arbitrary script with INTOFF in effect may cause unexpected results.

This change (together with other changes) serves mainly to allow asserting
that INTOFF is not in effect when starting the evaluation of a node.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

sh: Fix some dead stores.

Found by: clang static analyzer

sh: Fix more compiler warnings.

sh: Avoid overflow in atoi() when parsing HISTSIZE.

Side effect: a non-numeric HISTSIZE now results in the default size (100)
instead of 0.

sh: Prefer internal nextopt() to libc getopt().

This reduces code duplication and code size.

/usr/bin/printf is not affected.

Side effect: different error messages when certain builtins are passed
invalid options.

Add prototypes, ANSIfy functions definitions to reduce WARNS=6 output.

sh: Make various functions static.

sh: Fix duplicate prototypes for builtins.

Have mkbuiltins write the prototypes for the *cmd functions to builtins.h
instead of builtins.c and include builtins.h in more .c files instead of
duplicating prototypes for *cmd functions in other headers.

sh: Properly restore exception handler in fc.

If SIGINT arrived at exactly the right moment (unlikely), an exception
handler in a no longer active stack frame would be called.

Because the old handler was not used in the normal path, clang thought it
was a dead value and if an exception happened it would longjmp() to garbage.
This caused builtins/fc1.0 to fail if histedit.c was compiled with clang.

MFC after: 1 week

sh: Code size optimizations to "stack string" memory allocation:
* Prefer one CHECKSTRSPACE with multiple USTPUTC to multiple STPUTC.
* Add STPUTS macro (based on function) and use it instead of loops that add
nul-terminated strings to the stack string.

No functional change is intended, but code size is about 1K less on i386.

sh: Tweak some string constants to reduce code size.

* Reduce some needless differences.
* Shorten some error messages that should not happen.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Do not assume in growstackstr() that a "precious" character will be
immediately written into the stack after the call. Instead let the caller
manage the "space left".

Previously, growstackstr()'s assumption causes problems with STACKSTRNUL()
where we want to be able to turn a stack into a C string, and later
pretend the NUL is not there.

This fixes a bug in STACKSTRNUL() (that grew the stack) where:
1. STADJUST() called after a STACKSTRNUL() results in an improper adjust.
This can be seen in ${var%pattern} and ${var%%pattern} evaluation.
2. Memory leak in STPUTC() called after a STACKSTRNUL().

Reviewed by: jilles

In the spirit of r90111, depend on c89 and remove the "STATIC" macro
and its usage.

sh: Do not enter consecutive duplicates into the history.

This simply sets a flag in libedit. It has a shortcoming in that it does not
apply to multi-line commands.

Note that a configuration option for this is not going to happen, but always
having this seems better than not having it. NetBSD has done the same.

PR: bin/54683
Obtained from: NetBSD
MFC after: 1 month

sh: Add filename completion.

This uses the new libedit completion function with quoting support.

Unlike NetBSD, there is no 'set +o tabcomplete' option to disable
completion. I do not see any reason for such a special treatment, as
completion is rather useful and it is possible to do
bind ^I ed-insert
to disable completion and insert a tab character instead.

Submitted by: Guy Yur

sh: Pass TERM changes to libedit.

I have changed the patch slightly to ignore TERM changes in subshells.

PR: bin/146916
Submitted by: Guy Yur
Obtained from: NetBSD

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

sh: Various warning fixes (from WARNS=6 NO_WERROR=1):
- const
- initializations to silence -Wuninitialized (it was safe anyway)
- remove nested extern declarations
- rename "index" locals to "idx"

sh: Constify various strings.

Most of this is adding const keywords, but setvar() in var.c had to be
changed somewhat more.

sh: Some changes to stderr flushing:
* increase buffer size from 100 to 256 bytes
* remove implied flush from out2str(), in particular this avoids unnecessary
flushing in the middle of a -x tracing line
* rename dprintf() to out2fmt_flush(), make it flush out2 and use this
function in various places where flushing is desired after an error
message

sh: Improve handling of setjmp/longjmp volatile:
- remove ineffective and unnecessary (void) &var; [1]
- remove some unnecessary volatile keywords
- add a necessary volatile keyword
- save the old handler before doing something that could use the saved
value

Submitted by: Christoph Mallon [1]
Approved by: ed (mentor)

Fix the eval command in combination with set -e. Before this change the shell
would always terminate if eval returned with a non-zero exit status regardless
if the status was actually tested. Unfortunately a new file-scope variable
is needed, the alternative would only be to add a new parameter to all
built-ins.

PR: 134881

- Import the HEAD csup code which is the basis for the cvsmode work.

Commit the results of the typo hunt by Darren Pilgrim.
This change affects documentation and comments only,
no real code involved.

PR: misc/101245
Submitted by: Darren Pilgrim <darren pilgrim bitfreak org>
Tested by: md5(1)
MFC after: 1 week

Use the new name H_SETSIZE instead of the old H_EVENT to set the history
size.

PR: 86355

El_parse's third parameter is now const-qualified, add a cast.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

Delete worthless comments.

Set opterr to zero to avoid duplicate warnings from getopt(3) for unknown
options.

Call el_source() to read the contents of .editrc when line editing is enabled.

Obtained from: NetBSD
MFC after: 2 weeks

Add a `bind' builtin command, which is simply a wrapper around libedit's
builtin command of the same name. This allows the key bindings for the
shell's line editor to be changed.

MFC after: 2 weeks

Consistently use FBSDID

Fix typo causing ``fc -e'' to not work correctly. getopt() sets optarg,
not shoptarg.

Adding an entry to the history with H_ENTER moves libedit's internal
history cursor. Reset the cursor after adding the entry to the history
when doing ``fc -s'' so the output is correct.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.
o Change
int
foo() {
...
to
int
foo(void)
{
...

*** empty log message ***

Use PATH_MAX in preference in MAXPATHLEN.

Fix warnings, some of them serious because sh violated name
spaces reserved by the header files it includes.

mkinit.c still produces C code with redundant declarations, although
they are more harmless since they automatically derived from the right
places.

$Id$ -> $FreeBSD$

Add rcsid. Spelling.

When NO_HISTORY is defined and the history command is used
show an error message.

Inspired by: NetBSD

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Merge in NetBSD mods and -Wall cleaning.

Obtained from: NetBSD, me

Misc cleanups and fixes from Bruce:
- don't put \n on error() calls, error adds it already.
- don't prepend "ulimit" on error() calls in miscbltin.c.
- getopt typo on ulimit -p -> -u conversion
- get/setrlimit() calls were not being error checked

ulimit formatting cleanup from me, use same wording as bash on Bruce's
suggestion. Add ulimit arg to output on Joerg's suggestion.

Merge of 4.4-Lite2 sh source, plus some gcc -Wall cleaning. This is a
merge of parallel duplicate work by Steve Price and myself. :-]

There are some changes to the build that are my fault... mkinit.c was
trying (poorly) to duplicate some of the work that make(1) is designed to
do. The Makefile hackery is my fault too, the depend list was incomplete
because of some explicit OBJS+= entries, so mkdep wasn't picking up their
source file #includes.

This closes a pile of /bin/sh PR's, but not all of them..

Submitted by: Steve Price <steve@bonsai.hiwaay.net>, peter

Import the 4.4BSD-Lite2 /bin/sh sources

Requested by: joerg

(Note, this is mostly going to be conflicts, which is expected. Our entire
sh source has a mainline, so this should not change anything except for
a few new files appearing. I dont think they are a problem)

Remove trailing whitespace.

Reviewed by: phk

From "Philippe Charnier" <charnier@lirmm.fr>:

There is a bug in sh: the built in command "fc -l" generates
a core dump (*NULL in not_fcnumber).

According to the sh manual page (fc -l [-nr] [first [last]]), fc -l
is a correct sequence (in that case, values are defaulted to -16 and -1)
but fails when first is not given.

Added $Id$

BSD 4.4 Lite bin Sources