Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

sh: Fix heredoc at certain places in case and for

After an unescaped newline, there may be a here-document. Some places in
case and for did not check for one.

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D32628

sh/tests: Test some obscure cases with aliasing keywords

sh: Test that backslash-newline within single-quotes is not special

This works correctly, but the test may be helpful when modifying the parser.

sh: Fix use after free when resetting an in-use alias.

The special case of modifying an existing alias does not work correctly if
the alias is currently in use. Instead, handle this case by unaliasing the
old alias (if any) and then creating a new alias.

sh: Link the new test to the build.

sh: Add test for unaliasing an alias that is currently in use.

This already works correctly.

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

sh: Fix set -v and multiline history after r295825.

r295825 erroneously dropped the newline from the input data for 'set -v'
output and multiline history.

Reported by: vangyzen

sh: Rework code to remove '\0' from shell input.

This fixes bugs where '\0' was not removed correctly and speeds up the
parser.

sh: Add tests for comments in sh -c.

Fix build.

Sponsored by: The FreeBSD Foundation

First pass to fix the 'tests' packages.

Sponsored by: The FreeBSD Foundation

sh: Allow empty << EOF markers.

sh: Avoid negative character values from $'\Uffffffff' etc.

The negative value was not expected and generated the low 8 bits as a byte,
which may be an invalid character encoding.

The final shift in creating the negative value was undefined as well.

Make the temporary variable unsigned to fix this.

sh: Add test for $'\u' without any digits.

It is likely that $'\uXXXX' and $'\UXXXXXXXX' will allow fewer digits in
future. However, no digits at all should still be disallowed.

sh: Allow backslash-newline continuation in more places:

* directly after a $
* directly after ${
* between the characters of a multi-character operator token
* within a parameter name

sh: Add some tests for backslash-newline continuation.

sh: Remove arbitrary length limit on << EOF markers.

This also simplifies the code.

Convert bin/sh/tests to ATF

The new code uses a "test discovery mechanism" to determine
what tests are available for execution

The test shell can be specified via:

kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh

Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)

sh: Allow aliases to force alias substitution on the following word.

If an alias's value ends with a space or tab, the next word is also
checked for aliases.

This is a POSIX feature. It is useful with utilities like command and
nohup (alias them to themselves followed by a space).

sh: Add tests for alias names after another alias.

Since the first alias's value does not end with a blank, the next word
should not be checked for aliases.

sh: Add test for nested alias.

Migrate tools/regression/bin/ tests to the new layout.

This change is a proof of concept on how to easily integrate existing
tests from the tools/regression/ hierarchy into the /usr/tests/ test
suite and on how to adapt them to the new layout for src.

To achieve these goals, this change:

- Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/.
- Renames the previous regress.sh files to legacy_test.sh.
- Adds Makefiles to build and install the tests and all their supporting
data files into /usr/tests/bin/.
- Plugs the legacy_test test programs into the test suite using the new
TAP backend for Kyua (appearing in 0.8) so that the code of the test
programs does not have to change.
- Registers the new directories in the BSD.test.dist mtree file.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)