Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Separate dependencies on tools built for host.

When generated files depend on tools that need to be built for host,
we need to carefully separate them for the DIRDEPS_BUILD so we
only build them once.

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.

Fix building host tools for host

Several makefile depend on tools built for host.
At least when using DIRDEPS_BUILD we can build these for the
pseudo machine "host" to facilitate building on older host versions.

Ideally we would build these tools in their own directories to avoid
building more than needed.

For now, setting an appropriate default for BTOOLSPATH will suffice

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39708

awk: Enable tests again

Since we now pass all 24 of the NetBSD awk tests, re-enable these tests.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31370

awk: use awkgram.tab.h consistently

yacc makes awkgram.h. However, one true awk includes awkgram.tab.h, so
we link to for the builds. Make sure that we consistently link to it.
Also, restore the awkgram.tab.h dependency to maketab. It should not
have been deleted, despite apparently making meta build on stable/12
work. The important missing arc was proctab.c's dependence on
awkgram.tab.h.

MFC After: 1 day (build breakage)
Fixes: c50c8502cb629571f35089690d6e9a9bc4d60813
Sponsored by: Netflix

awk: Fix dependencies

proctab.c is generated from awktab.h, so needs to depend on it.
maketab does not depend on awktab.h, and gets the maketab.c dependency
automatically, so remove them both.

Normally, these don't matter. However, for a meta build, they can cause
us to build maketab twice (once host, once for target) resulting in a
binary that can't run on the host due to proctab.c racing maketab in
parallel legs. In stable/12, this was a reliably lost race, while in
main I've been unable to trigger the race at all (maybe due to dirdep
changes making main more robust).

MFC After: 1 day (build breakage)
Reported by: kp
Sponsored by: Netflix

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Fix .depend files to work for build tools.

This is somewhat of a follow-up to r335746.

MFC after: 2 weeks
Sponsored by: DellEMC

This should have been committed in r348511 with the awk update.

It was in my tree, the build worked, but I committed from contrib/one-true-awk
rather than the top level, so was omitted.

awk(1): Don't install tests at all

Tests were disconnected so that running `make check` in usr.bin/awk did not
have any effect, but CI runs use installed tests. Fully disconnect tests/
from the build for the time being as a short term solutio

Reported by: lwhsu

awk(1): Add necessary bits for connecting tests, but leave disconnected

The NetBSD test suite has 24 tests for awk, and we pass exactly 4 of them.
Add the necessary pieces for interested parties to easily connect the
tests and run them, but leave them disconnected for the time being.

Some of these tests outright segfault in our awk, others just exhibit the
wrong behavior.

Use SRCTOP to refer to awk source in contrib/awk and remove unnecessary AWKSRC prefix
for maketab.c

The former simplifies pathing in make/displayed output, whereas the latter was just
unnecessarily superfluous since .PATH referenced the path to maketab.c earlier on in
the Makefile.

MFC after: 1 week
Sponsored by: Dell EMC Isilon

WITH_META_MODE: Fix rebuilding maketab outside of build-tools.

The bsd.dep.mk yacc targets rely on only the .c file getting a .meta
file. However the previous code here relying on only the .h file meant
that it would be generated with a .meta file. r301285 made it so that
the .h file is never expected to get a .meta file. To keep this
restriction in place add in an extra dependency on the .c file so that
it is generated at this time. It's a hack but the best for the patterns
we have at the moment for handling build-tools and side-effect-generated
files.

Reported by: Mark Millard
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Run the staged bootstrap-tools version of build-tools.

This avoids running target binaries.

Sponsored by: EMC / Isilon Storage Division

META_MODE: Don't rebuild build-tools targets during normal build.

This avoids 'build command changed' due to CFLAGS/CC changes during the
normal build. Without this the build-tools targets end up rebuilding
for the *target* rather than keeping the native versions built in
build-tools.

Sponsored by: EMC / Isilon Storage Division

META MODE: Don't create .meta files when symlinking sources into the obj directory.

Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.

Sponsored by: EMC / Isilon Storage Division

Convert to usr.bin/ to LIBADD
Reduce overlinking

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

Allow awk (the one true one!) to handle 64 files instead of just 20.
The current FreeBSD syscall generation script uses all 20 and I need
another open file.

It's a shame that something named as the 'one-true-awk' is so limited
by an old denition like FOPEN_MAX when it could just make the file
handling dynamic.

This is done to avoid touching contrib sources on a vendor branch.

Now that we're way past our conversion from GNU awk to BWK awk, we don't
need to primarily install as a non-conflicting name.

The AWK 23-Oct-2007 release includes fixes for FreeBSD PR's bin/104795
and bin/100443.

Fix a bug converting a variable from the numeric type to a string.

PR: bin/104795
MFC after: 3 days

Initial memory allocation for fields was off-by-one.

PR: bin/100443
MFC after: 3 days

Add temporary patches to make one-true-awk respect locale's collating order
in [a-z] bracket expressions, until a more complete fix (like handing BREs)
is ready.

Prodded by: ache
OK'ed by: tjr

Use
cat ${.ALLSRC} > ${.TARGET}
rather than
ln -sf ${.ALLSRC} ${.TARGET}
not to depends on absolute-path of symbolic links.

Commented by: marcel, obrien, bde

For the bwk_20030314 version, we have to -DHAS_ISBLANK now.
Oh how I wish the author had accepted my "#ifndef isblank" patch instead.

BWK awk appears to work for world builds just fine now. We want to make
sparc64 the same as the other platforms -- so do it by moving the other
platforms toward the sparc64.

Style.

Style.

Make this compile in a standalone environment.
Tidy up CLEANFILES.
Use built-in rules for building "maketab".

Use our auto-YACC'ing rules to do most of the work.

Submitted by: bde

To quote BDE: "the k part of awk should understand the ctype macros and not
need the -funsigned-char hack."

Also add needed CLEANSFILE.

Simpler way to handling the names needed on the various platforms.

On the sparc64 platform we install the `1 true AWK' as "awk". On all other
platforms we install as "nawk".

At the moment Gawk is borked on sparc64, however BWK AWK works fine.

Switch to the One True AWK from Bell-Labs, away from GNU AWK.

Fix .depend files to work for build tools.

This is somewhat of a follow-up to r335746.

MFC after: 2 weeks
Sponsored by: DellEMC

This should have been committed in r348511 with the awk update.

It was in my tree, the build worked, but I committed from contrib/one-true-awk
rather than the top level, so was omitted.

awk(1): Don't install tests at all

Tests were disconnected so that running `make check` in usr.bin/awk did not
have any effect, but CI runs use installed tests. Fully disconnect tests/
from the build for the time being as a short term solutio

Reported by: lwhsu

awk(1): Add necessary bits for connecting tests, but leave disconnected

The NetBSD test suite has 24 tests for awk, and we pass exactly 4 of them.
Add the necessary pieces for interested parties to easily connect the
tests and run them, but leave them disconnected for the time being.

Some of these tests outright segfault in our awk, others just exhibit the
wrong behavior.

Use SRCTOP to refer to awk source in contrib/awk and remove unnecessary AWKSRC prefix
for maketab.c

The former simplifies pathing in make/displayed output, whereas the latter was just
unnecessarily superfluous since .PATH referenced the path to maketab.c earlier on in
the Makefile.

MFC after: 1 week
Sponsored by: Dell EMC Isilon

WITH_META_MODE: Fix rebuilding maketab outside of build-tools.

The bsd.dep.mk yacc targets rely on only the .c file getting a .meta
file. However the previous code here relying on only the .h file meant
that it would be generated with a .meta file. r301285 made it so that
the .h file is never expected to get a .meta file. To keep this
restriction in place add in an extra dependency on the .c file so that
it is generated at this time. It's a hack but the best for the patterns
we have at the moment for handling build-tools and side-effect-generated
files.

Reported by: Mark Millard
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Run the staged bootstrap-tools version of build-tools.

This avoids running target binaries.

Sponsored by: EMC / Isilon Storage Division

META_MODE: Don't rebuild build-tools targets during normal build.

This avoids 'build command changed' due to CFLAGS/CC changes during the
normal build. Without this the build-tools targets end up rebuilding
for the *target* rather than keeping the native versions built in
build-tools.

Sponsored by: EMC / Isilon Storage Division

META MODE: Don't create .meta files when symlinking sources into the obj directory.

Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.

Sponsored by: EMC / Isilon Storage Division

Convert to usr.bin/ to LIBADD
Reduce overlinking

Check in first src/tests snapshot from NetBSD anoncvs

Sources were obtained like so:

% export CVSROOT="anoncvs@anoncvs.NetBSD.org:/cvsroot"
% cvs -z9 co -D "09/30/2014 20:45" -P src/tests
% mv src/tests/* tests/dist/.

'*CVS*' has been added to svn:ignore to ease updating periodically from
upstream

Some line ending issues had to be resolved with test outputs and scripts
via dos2unix and by deleting the eol-style property set in usr.bin/sort

Discussed with: rpaulo
Sponsored by: EMC / Isilon Storage Division

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Update to a 26-Nov-2009 release.

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

- Import the HEAD csup code which is the basis for the cvsmode work.

Allow awk (the one true one!) to handle 64 files instead of just 20.
The current FreeBSD syscall generation script uses all 20 and I need
another open file.

It's a shame that something named as the 'one-true-awk' is so limited
by an old denition like FOPEN_MAX when it could just make the file
handling dynamic.

This is done to avoid touching contrib sources on a vendor branch.

Now that we're way past our conversion from GNU awk to BWK awk, we don't
need to primarily install as a non-conflicting name.

The AWK 23-Oct-2007 release includes fixes for FreeBSD PR's bin/104795
and bin/100443.

Fix a bug converting a variable from the numeric type to a string.

PR: bin/104795
MFC after: 3 days

Initial memory allocation for fields was off-by-one.

PR: bin/100443
MFC after: 3 days

Add temporary patches to make one-true-awk respect locale's collating order
in [a-z] bracket expressions, until a more complete fix (like handing BREs)
is ready.

Prodded by: ache
OK'ed by: tjr

Use
cat ${.ALLSRC} > ${.TARGET}
rather than
ln -sf ${.ALLSRC} ${.TARGET}
not to depends on absolute-path of symbolic links.

Commented by: marcel, obrien, bde

For the bwk_20030314 version, we have to -DHAS_ISBLANK now.
Oh how I wish the author had accepted my "#ifndef isblank" patch instead.

BWK awk appears to work for world builds just fine now. We want to make
sparc64 the same as the other platforms -- so do it by moving the other
platforms toward the sparc64.

Style.

Style.

Make this compile in a standalone environment.
Tidy up CLEANFILES.
Use built-in rules for building "maketab".

Use our auto-YACC'ing rules to do most of the work.

Submitted by: bde

To quote BDE: "the k part of awk should understand the ctype macros and not
need the -funsigned-char hack."

Also add needed CLEANSFILE.

Simpler way to handling the names needed on the various platforms.

On the sparc64 platform we install the `1 true AWK' as "awk". On all other
platforms we install as "nawk".

At the moment Gawk is borked on sparc64, however BWK AWK works fine.

Switch to the One True AWK from Bell-Labs, away from GNU AWK.