usr.bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

indent(1): add fallthrough markers

This silences -Wimplicit-fallthrough warnings.

Submitted by: Michael Paquier
Obtained from: postgresql.org
MFC after: 3 days

indent: fix the -fno-common build

Spread the globals far and wide, hopefully to the files that make the most
sense.

-fno-common will become the default in GCC10/LLVM11.

MFC after: 3 days

capsicum: use a new capsicum helpers in tools

Use caph_{rights,ioctls,fcntls}_limit to simplify the code.

indent(1): rewrite the integer/floating constant scanning part of lexi.c

Remove procedural code that did the scanning, which was faulty and didn't
support complex constants such as 0x1p-61. Replace it with a finite state
machine expressed as a transition table. The table was rewritten by hand
from lx's output, given parts of grammar expressed as regular expressions.

lx is Katherine Flavel's lexer generator, currently available at
https://github.com/katef/libfsm and the parts of grammar were taken from
http://quut.com/c/ANSI-C-grammar-l-2011.html and extended to support binary
integer constants which are a popular GCC extension.

Reported by: bde

Convert `cap_enter() < 0 && errno != ENOSYS` to `caph_enter() < 0`.

No functional change intended.

indent(1): group global option variables into an options structure

It's clearer now when a variable represents a toggable command line option.

Many options were stored in the parser's state structure, so fix also that.

indent(1): remove static const char copyright[]

It repeats what is already said in the heading comment and it's optimized out
so serves no purpose.

indent(1): replace BSD bcopy() with C memmove()

indent(1): new option -lpl

With -lpl, code surrounded by parentheses in continuation lines is lined up
even if it would extend past the right margin.

With -nlpl (the default), such a line that would extend past the right
margin is moved left to keep it within the margin, if that does not require
placing it to the left of the prevailing indentation level.

These switches have no effect if -nlp is selected.

Submitted by: Tom Lane

indent(1): revert introduction of -lpl

That was committed with the wrong message. Will be re-added.

indent(1): new option -lpl (always line up to parenthesis)

With -lp, if a line has an opening paren which is not closed on that line,
then continuation lines will be lined up to start at the character position
just after the opening paren.

Submitted by: Tom Lane

indent(1): if the token is a "[" then neither of the blocks is relevant

indent(1): don't add a space after a label

It's not needed and it fools pr_comment().

indent(1): don't overflow di_stack[]

indent(1): improve CHECK_SIZE_ macros

Rewrite the macros so that they take a parameter. Consumers use it to signal
how much room in the buffer they need; this lets them do that once when
required space is known instead of doing the check once every loop step.

Also take the parameter value into consideration when resizing the buffer;
the requested space may be larger than the constant 400 bytes that the
previous version used - now it's the sum of those two values.

On the consumer side, don't copy strings byte by byte - use memcpy().

Deduplicate code that copied base 2, base 8 and base 16 literals.

Don't advance the e_token pointer once the token has been copied into
s_token. This allows easy calculation of the token's length.

indent(1): remove troff output support

The troff output in indent was invented at Sun and the online documentation
for some post-SunOS operating system includes this:
The usual way to get a troffed listing is with the command
indent -troff program.c | troff -mindent

The indent manual page in FreeBSD 1.0 already lacks that information and
troff -mindent complains about not being able to find the macro file.
It seems that the file did exist on SunOS and was supposed to be imported
into 4.3BSD together with the feature, but that has never happened.

Removal of troff output support simplifies a lot of indent's code.

vgrind(1) seems to be a promising replacement.

indent(1): the check for buffer overflow has to be done before copy

indent(1): use errx() instead of abort()

indent(1): limit character classification functions' input to unsigned char

indent(1): don't format function declarations as variables

indent(1): remove is_procname.

It was a shorthand for checking if ps.procname is a non-empty string; the
same can be done with ps.procname[0] which avoids the need for updating
is_procname after every call to lexi().

indent(1): disjoint parser state from lexi()

The function is sometimes used as a look-ahead, so ideally it should bear
no information about parser state.

indent(1): improve handling of boxed comments indentation

The trick is to copy everything from the start of the line into the buffer
that stores newlines and comments until indent finds a brace or an else.
pr_comment() will use that information to calculate the original indentation
of the boxed comment.

This requires storing two pieces of information: the real start of the
buffer (sc_buf) and the start of the comment (save_com).

indent(1): improve predictability of lexi()

lexi() reads the input stream and categorizes the next token. indent will
sometimes buffer up a sequence of tokens in order rearrange them. That is
needed for properly cuddling else or placing braces correctly according to
the chosen style (KNF vs Allman) when comments are around. The loop that
buffers tokens up uses lexi() to decide if it's time to stop buffering. Then
the temporary buffer is used to feed lexi() the same tokens again, this time
for normal processing.

The problem is that lexi() apart from recognizing the token, can change
a lot of information about the current state, for example ps.last_nl,
ps.keyword, buf_ptr. It also abandons leading whitespace, which is needed
mainly for comment-related considerations. So the call to lexi() while
tokens are buffered up and categorized can change the state before they're
read again for normal processing which may easily result in changing
interpretation of the current state and lead to incorrect output.

To work around the problems:
1) copy the whitespace into the save_com buffer so that it will be read
again when processed
2) trick lexi() into modifying a temporary copy of the parser state instead
of the original.

indent(1): improve handling of comments and newlines between "if (...)" or
"while (...)" and "else" or "{"

* Don't flush newlines - there can be multiple of them and they can happen
before a token that isn't else or {. Instead, always store them in save_com.
* Don't dump the buffer's contents on newline assuming that there is only
one comment before else or {.
* Avoid producing surplus newlines, especially before else when -ce is on.
* When -bl is on, don't treat { as a comment (was implemented by falling
through "case lbrace:" to "case comment:").

This commit fixes the above, but exposes another bug and thus breaks several
other tests. Another commit will make them pass again.

indent(1): restore working -pcs

My previous indent(1) commit accidentally broke the -pcs option (which adds
space between function name and opening parenthesis in function calls) by
copying all but one of a few conditions in an if clause. Reinstate the
condition.

Add a regression test to lower the chances of breaking it again.

Correct a comment with description of what the option does.

indent(1): don't add unneeded space to function pointer declarations

If the current token is an opening parenthesis, it's either a function call
(or sizeof or offsetof) or a declaration. The former doesn't need a space
before the parenthesis.

indent(1): don't indent typedef declarations as object declarations

indent(1): consider tab characters when forcing a newline after a comma

indent(1): identifiers inside parentheses are not declarations

Also make lparen position calculation consider tab stops.

This improves function pointer typedef formatting.

spdx: initial adoption of licensing ID tags.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133

Respect SIMPLE_BACKUP_SUFFIX environment variable in indent(1)

Instead of using a non-configurable ".BAK" suffix, respect the
SIMPLE_BACKUP_SUFFIX environment variable also used by patch(1). This
simplifies cleanup operations in some patch/indent workflows.

Reviewed by: cem (earlier version), emaste, pstef
Approved by: emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D10921

indent(1): add option -tsn for setting tab size.

indent(1): don't produce unneeded space character in function declarators.

indent(1): add option -P for loading user-provided files as profiles

Without this change, indent(1) would only look to load options from ~/.indent.pro if it's there and -npro wasn't used on the command line. This option lets the user set their own path to the file.

Approved by: pfg (mentor)
Differential Revision: https://reviews.freebsd.org/D9010

indent(1): Avoid out of bounds access of array ps.paren_indents

ps.p_l_follow can't be allowed to grow beyond maximum index of paren_indents.

Approved by: pfg (mentor)

indent(1): Fix indent's confusion about custom FreeBSD macros.

Teach indent(1) about storage-class specifiers. Don't assume
"in_parameter_declaration" state if "in_decl" hasn't been set. Don't set
"in_decl" for storage-class specifiers.

That set of changes helps with recognizing the difference between file
scope declarations like this:

static LIST_HEAD(, alq) ald_active;
static int ald_shuttingdown = 0;
struct thread *ald_thread;

and old style function declarators like this:

static int
do_execve(td, args, mac_p)
struct thread *td;
struct image_args *args;
struct mac *mac_p;
{

Unfortunately, at the same time this change makes indent(1) require
explicit int in declarations like "static a;", in order to understand that
it's part of a declaration. On the other hand, declarations like in the
first example are no longer indented as if ald_shuttingdown and ald_thread
were parameters of a function named LIST_HEAD.

Submitted by: Piotr Stefaniak

indent(1): Don't ignore newlines after comments that follow braces.

indent.c has a special loop that stores tokens from between an if () and
the next statement into a buffer. The loop ignored all newlines, but that
resulted in not calling dump_line() when it was needed to produce the
final line of the buffered up comment.

Taken from: Piotr Stefaniak

indent(1): Properly handle the wide string literal and wide char constant L.

indent(1) treated the "L" in "L'a'" as if it were an identifier and forced
a space character after it, breaking valid code.

PR: 143090
MFC after: 2 weeks

Move sys/capsicum.h includes after types.h or param.h

This is not actually documented or even implied in style(9). Make the change
to match convention. Someone should document this convention in style(9).

Reported by: jhb
Sponsored by: EMC Dell Isilon

indent(1): Capsicumify

This is a nice and trivial program for sandboxing. One input file, one
output file.

Reviewed by: pfg
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D7920

indent(1): remove dead assignments.

Taken from: Piotr Sephaniak

indent(1): Use bsearch() for looking up type keywords.

Reference:
https://github.com/pstef/freebsd_indent/commit/f3b8e6e57fd47364b2360b44011a2fad5d11adc7

Submitted by: Piotr Stefaniak
Differential Revision: https://reviews.freebsd.org/D6966

indent(1): add new -sac and -U options.

Add -sac (space after cast) and -nsac options.
These control whether space character is put after a cast operator or not.
Default is -nsac.

Add -U option for providing a file containing list of types.
This is needed for properly deciding which asterisks denote unary
operation and which denote binary.

These come from PostgreSQL.

Reference:
https://github.com/pstef/freebsd_indent/commit/84b00e3d46dfd6d955b2f481a1f3b275de9ad6d1
https://github.com/pstef/freebsd_indent/commit/49c52cf383fa2a246a1a22c6640a5a21b0f1fd90

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Submitted by: Piotr Stefaniak

indent(1): accept offsetof(3) as a keyword.

Reference:
https://github.com/pstef/freebsd_indent/commit/c470e5e2c974aa38450ca4762b93829f7a7bfa4d

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Submitted by: Piotr Stefaniak

indent(1): Use a dash in the license headers.

Use of the canonical dash avoids indent(1) from reformatting the
license headers.

indent(1): Don't newline on cpp lines like #endif unless -bacc is on.

Reference:
https://github.com/pstef/freebsd_indent/commit/01f36f4141c71754b3a73a91886fb425bab0df3e

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Submitted by: Piotr Stefaniak

indent(1): Consistently indent declarations.

This fixes a very visible issue that may be hidden by some indent.pro
settings as in the example from FreeBSD's /usr/share.

From Piotr's log:
____
To prevent losing tabs from indentation in declarations, FreeBSD indent's
r125624 added code for the most common case when it's an identifier that
is indented, but didn't do anything with the original code that did the
same for any other cases. The other cases are: lparens (function pointer
declaration), asterisks (pointer declaration), stray semicolons, and
commas leading identifiers instead of trailing them.

Use the code added in r125624 (and improved in later commits) to write a
new function indent_declaration() and use it in all places that meant to
indent declarations. In order to indent only once per line, reuse existing
ps.dumped_decl_indent variable that was only used when formatting for
troff (-troff) until now.
____

Reference:
https://github.com/pstef/freebsd_indent/commit/ddd263db2a59978f43468989eff65299cf3ce7e1

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Submitted by: Piotr Stefaniak

Use nitems() from sys/param.h.

MFC after: 2 weeks.
Sponsored by: gandi.net (BSD Day Taiwan)

indent(1): Yet more style issues.

strchr(3) returns a pointer not a boolean.
Attempt to make the style somewhat more ocnsistent with what indent
had before recent changes.

Pointed out by: bde

indent(1): Attempt to preserve some consistent style.

Remove the excessive braces from r303485 and align the comments to the
right as done in the rest of the code. This is not nice but there is no
clear way to make it nice (and KNF).

Pointed out by: bde

indent(1): Use NULL instead of zero for pointers.

indent(1): fix struct termination detection.

Semicolons inside struct declarations don't end the declarations.

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Obtained from: Piotr Stefaniak

indent(1): Fix breakage caused by single comment following "else".

indent(1) simply wasn't taught that "else" may be followed by a comment
without any opening brace anywhere on the line, so it was very confused
in such cases.

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Obtained from: Piotr Stefaniak

indent(1): Avoid out of bound access of array codebuf.

dump_line() requires s_code to be a string, because it will call count_spaces().

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Obtained from: Piotr Stefaniak

indent: avoid calling write(2) with a negative second parameter.

negative_returns: n is passed to a parameter that cannot be negative.
Fix a style issue while here.

CID: 1008107

Spelling fixes for usr.bin/

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Use NULL instead of 0 when setting up pointer.

MFC r205988: indent(1): correctly handle case/label at the very start of a function

indent(1): correctly handle case/label at the very start of a function

Obtained from: NetBSD (rev. 1.11 of indent.c)
MFC after: 1 week

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

- Import the HEAD csup code which is the basis for the cvsmode work.

initialize variable (WARNS=6)

Declare find_err in indent_globs.h where the other shared variables
live rather than listing it as an extern in indent.c.

Fix problems with non-8 space tabs. New options for function
declarations with the opening brace on the same line as the declaration
of arguments all spaces and no tabs (a feature which exists in GNU's
indent). Man page update to follow RSN.

PR: bin/67983
Submitted by: Chip Norkus <wd@teleri.net>
Style guidance and bug for bug compatibility by: bde
MFC after: 2 weeks

Added an option -ldi<N> to control indentation of local variable names.
The default is to be backwards compatible and non-KNF (use the same
indentation for locals as for globals; -ldi0 gives KNF indentation
for locals (none)). The indentation for globals also applies to struct
member names in local declatations. The indentation of variable names
in multi-line declarations is broken in various ways and this commit
gives some new variations.

indent.1:
Also clarified the description of -di<N>.

Only use tabs to indent variable names if the declaration indent is
nonzero (so that the 1-char indentation given by -di0 is never rendered
by a tab).

Removed garbage commented out code for setting the indentation of variable
names.

Backed out a part of previous commit that wasn;t mentioned in the log
message and wasn't quite ready (it avoided indenting the names of local
variables).

Fixed tab lossage in indented variable names. The -diN option was
worse than useless because it caused things like mangling of
"^int\tfoo" to "int foo" (this for N = 8). This quick fix breaks
the invariant that characters between s_code and e_code have width 1,
but nothing seems to depend on this.

err() on allocation failure. WARNS=9 compliant
use #if 0, #ifndef lint, #endif /* not lint */, #endif ordering
when a message is provided, use errx() instead of err().

Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup,
especially in troff files.

Spelling

Consistently use FBSDID

Remove deprecated register qualifier.

1) Const enough things to avoid warnings.
2) Cast ifdef_level to a size_t before comparing it to a ratio of size_ts.
Ifdef_level should always be positive.
3) Complete prototype for chfont.
4) Cast some ptrdiff_ts to ints before using as a field width.
5) Avoid shadowing a local variable p with another local variable p.

Make this compile cleanly when warnings are enabled:
- ANSIfy function declarations
- braces around initializers structs within structs
- add parens in complicated expressions
- disambiguate dangling elses
- no more implicit int
- make functions static where possible
- use prototypes
- don't use varargs hack for diag()

Requested by: joerg
MFC after: 2 weeks

Fix indent endless on garbage input (i.e., if it encountered EOF while
waiting for a '}' nesting terminator)

Obtained from: OpenBSD rev 1.8

Preceed/preceeding are not english words. Use precede and preceding.

Don't abuse the SCCS `@(#)' for RCS.

Requested by: bde

Add or fix FreeBSD IDs.

"Implement -[n]fcb (formatting of block comments) and attempt to implement
no-space=after-sizeof (not optional) and no-space-after 'struct foo *'
(not optional). Without these, indent unKNFizes even more perfectly KNF code."

Submitted by: bde

Don't call err() with no format string.

$Id$ -> $FreeBSD$

Clean up some ambiguous nested if/elses.

PR: 7583
Submitted by: Dave Glowacki <dglo@SSEC.WISC.EDU>
Support input via stdin if no input file name specified explicitly

Use err(3) instead of local redefinition. Add usage().

BSD 4.4 Lite Usr.bin Sources