awk: Merge in bsd-feature branch of OTA from 20240422 (31bb33a32f71)

In the last 2nd edition import, I mistakenly grabbed from the 'main'
branch of upstream rather than the bsd-feature branch. This means that
we have a regression in awk from that point forward: all the
BSD-specific bit functions (and a few others) were dropped. This
restores it at the same level.

MFC After: 1 day
Sponsored by: Netflix

ota: Merge one true awk 20240422 (a3b68e649d2d)

Apr 22, 2024:
fixed regex engine gototab reallocation issue that was
introduced during the Nov 24 rewrite. Thanks to Arnold Robbins.
Fixed a scan bug in split in the case the separator is a single
character. thanks to Oguz Ismail for spotting the issue.

Mar 10, 2024:
fixed use-after-free bug in fnematch due to adjbuf invalidating
the pointers to buf. thanks to github user caffe3 for spotting
the issue and providing a fix, and to Miguel Pineiro Jr.
for the alternative fix.
MAX_UTF_BYTES in fnematch has been replaced with awk_mb_cur_max.
thanks to Miguel Pineiro Jr.

Sponsored by: Netflix

Merge one true awk from 2024-01-22 for the Awk Second Edition support

This brings in Unicode support, CSV support and a number of bug fixes.
They are described in _The AWK Programming Language_, Second Edition, by
Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 2024,
ISBN-13 978-0138269722, ISBN-10 0138269726).

Sponsored by: Netflix

Revert "awk: Merge upstream 2nd Edition Awk Book"

The pre-push testing I did turned out to be testing the old version with
the old testsuite (for reasons I don't understnad). There's issues with
the new version, the new test in the suite or (likely) both. Revert
until they can be chased down.

This should also fix the github CI that's gone red since this commit.

This reverts commit 3fd60a6b73ac01a72df89751f173970fae4cae73, reversing
changes made to 194df014feebd8b169b41ecd75ae73d63a792d6b.

Sponsored by: Netflix

awk: Merge upstream 2nd Edition Awk Book

Merge in the November 2nd, 2023 version of one true awk.

This brings in Unicode support, CSV support and a number of bug fixes.

Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42447

awk: errror on printf format strings lacking conversion specifier

Reported by: phk
Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39573

awk: Remove last markings we have on awk

We normally don't add $FreeBSD$ to contrib software. However, these
changes date back to the CVS era of source code management and have been
overlooked. Now that all these files are back to the same as the
upstream bsd-features branch, remove the FreeBSD specific changes, which
are now just $FreeBSD$ and the (FreeBSD) in the version string.

MFC After: 2 weeks
Sponsored by: Netflix

awk: revert to upstream behavior for ranges for gawk compatibility

In 2005, FreeBSD changed one-true-awk to honor the locale's collating
order. This was billed as a temporary patch. It was also compatible with
the then-current behavior of gawk. That temporary patch has lasted 16
years now.

However, IEEE Std 1003.1-2008 changed the behaivor of ranges in regular
expressions outside of the "C" and "POSIX" locales to be undefined.

Starting in 2011, gawk 4.0 stopped using the locale for the range
regular expressions and used the traditional behavior only. The
maintainer had grown weary of answering why '[A-Z]' would sometimes
match lower-case expressions. The details about are explained here:
https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html

To restore compatibility with other implementaitons of awk, revert this
patch. FreeBSD is the odd-system out. It also has the nice side effect
of eliminating the last of our differences with upstream one-true-awk.

Reviewed by: cy, rgrimes
MFC After: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31114

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Another partial revert of r301289.

In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.

Fix mismerge that crept into r301289.

The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.

Reapply r301289 by pfg:

| MFV r300961: one-true-awk: replace 0 with NULL for pointers
| Also remove a redundant semicolon.
| Also had to rebase on upstream pull.

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Fix %c for floating values that become 0 when coerced to int.

Obtained from: OpenBSD run.c 1.36 (From Jeremy Devenport)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Implement gawk multiple-arg extension to and, or, and xor.

gawk allows multiple arguemnts to bit-wiste and, or and xor
functions. Implement an arbitrary number of arguments for these
functions. Also, use NULL in preference to 0 to match rest of file.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12361

Bring in bit operation functions, ala gawk.

These are from OpenBSD:
>>> Extend awk with bitwise operations. This is an extension to the awk
>>> spec and documented as such, but comes in handy from time to time.
>>> The prototypes make it compatible with a similar GNU awk extension.
>>>
>>> ok millert@, enthusiasm from deraadt@

Edited to fix cut and paste in error messages, as well as
using tabs instead of spaces after #defines added.

Obtained From: OpenBSD awk.h 1.12, lex.c 1.10, run.c 1.29
Differential Revision: https://reviews.freebsd.org/D12361
Sponsored by: Netflix

MFV r300961:
one-true-awk: replace 0 with NULL for pointers

Also remove a redundant semicolon.

awk: Use random(3) instead of rand(3)

While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).

Use random(3) for awk as is done in other systems.

Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.

PR: 193147
MFC after: 5 weeks

MFV: one-true-awk 20121220.

MFC after: 1 month

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 7-Feb-2004 release.

Vendor import of bwk's 29-Jul-2003 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 13-Dec-2002 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Vendor import of bwk's 2002-02-18 release. Most significant update is the
inclusion of my character class patch.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

ota: Merge one true awk 20240422 (a3b68e649d2d)

Apr 22, 2024:
fixed regex engine gototab reallocation issue that was
introduced during the Nov 24 rewrite. Thanks to Arnold Robbins.
Fixed a scan bug in split in the case the separator is a single
character. thanks to Oguz Ismail for spotting the issue.

Mar 10, 2024:
fixed use-after-free bug in fnematch due to adjbuf invalidating
the pointers to buf. thanks to github user caffe3 for spotting
the issue and providing a fix, and to Miguel Pineiro Jr.
for the alternative fix.
MAX_UTF_BYTES in fnematch has been replaced with awk_mb_cur_max.
thanks to Miguel Pineiro Jr.

Sponsored by: Netflix

Merge one true awk from 2024-01-22 for the Awk Second Edition support

This brings in Unicode support, CSV support and a number of bug fixes.
They are described in _The AWK Programming Language_, Second Edition, by
Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 2024,
ISBN-13 978-0138269722, ISBN-10 0138269726).

Sponsored by: Netflix

Revert "awk: Merge upstream 2nd Edition Awk Book"

The pre-push testing I did turned out to be testing the old version with
the old testsuite (for reasons I don't understnad). There's issues with
the new version, the new test in the suite or (likely) both. Revert
until they can be chased down.

This should also fix the github CI that's gone red since this commit.

This reverts commit 3fd60a6b73ac01a72df89751f173970fae4cae73, reversing
changes made to 194df014feebd8b169b41ecd75ae73d63a792d6b.

Sponsored by: Netflix

awk: Merge upstream 2nd Edition Awk Book

Merge in the November 2nd, 2023 version of one true awk.

This brings in Unicode support, CSV support and a number of bug fixes.

Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42447

awk: errror on printf format strings lacking conversion specifier

Reported by: phk
Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39573

awk: Remove last markings we have on awk

We normally don't add $FreeBSD$ to contrib software. However, these
changes date back to the CVS era of source code management and have been
overlooked. Now that all these files are back to the same as the
upstream bsd-features branch, remove the FreeBSD specific changes, which
are now just $FreeBSD$ and the (FreeBSD) in the version string.

MFC After: 2 weeks
Sponsored by: Netflix

awk: revert to upstream behavior for ranges for gawk compatibility

In 2005, FreeBSD changed one-true-awk to honor the locale's collating
order. This was billed as a temporary patch. It was also compatible with
the then-current behavior of gawk. That temporary patch has lasted 16
years now.

However, IEEE Std 1003.1-2008 changed the behaivor of ranges in regular
expressions outside of the "C" and "POSIX" locales to be undefined.

Starting in 2011, gawk 4.0 stopped using the locale for the range
regular expressions and used the traditional behavior only. The
maintainer had grown weary of answering why '[A-Z]' would sometimes
match lower-case expressions. The details about are explained here:
https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html

To restore compatibility with other implementaitons of awk, revert this
patch. FreeBSD is the odd-system out. It also has the nice side effect
of eliminating the last of our differences with upstream one-true-awk.

Reviewed by: cy, rgrimes
MFC After: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31114

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Another partial revert of r301289.

In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.

Fix mismerge that crept into r301289.

The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.

Reapply r301289 by pfg:

| MFV r300961: one-true-awk: replace 0 with NULL for pointers
| Also remove a redundant semicolon.
| Also had to rebase on upstream pull.

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Fix %c for floating values that become 0 when coerced to int.

Obtained from: OpenBSD run.c 1.36 (From Jeremy Devenport)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Implement gawk multiple-arg extension to and, or, and xor.

gawk allows multiple arguemnts to bit-wiste and, or and xor
functions. Implement an arbitrary number of arguments for these
functions. Also, use NULL in preference to 0 to match rest of file.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12361

Bring in bit operation functions, ala gawk.

These are from OpenBSD:
>>> Extend awk with bitwise operations. This is an extension to the awk
>>> spec and documented as such, but comes in handy from time to time.
>>> The prototypes make it compatible with a similar GNU awk extension.
>>>
>>> ok millert@, enthusiasm from deraadt@

Edited to fix cut and paste in error messages, as well as
using tabs instead of spaces after #defines added.

Obtained From: OpenBSD awk.h 1.12, lex.c 1.10, run.c 1.29
Differential Revision: https://reviews.freebsd.org/D12361
Sponsored by: Netflix

MFV r300961:
one-true-awk: replace 0 with NULL for pointers

Also remove a redundant semicolon.

awk: Use random(3) instead of rand(3)

While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).

Use random(3) for awk as is done in other systems.

Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.

PR: 193147
MFC after: 5 weeks

MFV: one-true-awk 20121220.

MFC after: 1 month

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 7-Feb-2004 release.

Vendor import of bwk's 29-Jul-2003 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 13-Dec-2002 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Vendor import of bwk's 2002-02-18 release. Most significant update is the
inclusion of my character class patch.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

Merge one true awk from 2024-01-22 for the Awk Second Edition support

This brings in Unicode support, CSV support and a number of bug fixes.
They are described in _The AWK Programming Language_, Second Edition, by
Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 2024,
ISBN-13 978-0138269722, ISBN-10 0138269726).

Sponsored by: Netflix

Revert "awk: Merge upstream 2nd Edition Awk Book"

The pre-push testing I did turned out to be testing the old version with
the old testsuite (for reasons I don't understnad). There's issues with
the new version, the new test in the suite or (likely) both. Revert
until they can be chased down.

This should also fix the github CI that's gone red since this commit.

This reverts commit 3fd60a6b73ac01a72df89751f173970fae4cae73, reversing
changes made to 194df014feebd8b169b41ecd75ae73d63a792d6b.

Sponsored by: Netflix

awk: Merge upstream 2nd Edition Awk Book

Merge in the November 2nd, 2023 version of one true awk.

This brings in Unicode support, CSV support and a number of bug fixes.

Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42447

awk: errror on printf format strings lacking conversion specifier

Reported by: phk
Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39573

awk: Remove last markings we have on awk

We normally don't add $FreeBSD$ to contrib software. However, these
changes date back to the CVS era of source code management and have been
overlooked. Now that all these files are back to the same as the
upstream bsd-features branch, remove the FreeBSD specific changes, which
are now just $FreeBSD$ and the (FreeBSD) in the version string.

MFC After: 2 weeks
Sponsored by: Netflix

awk: revert to upstream behavior for ranges for gawk compatibility

In 2005, FreeBSD changed one-true-awk to honor the locale's collating
order. This was billed as a temporary patch. It was also compatible with
the then-current behavior of gawk. That temporary patch has lasted 16
years now.

However, IEEE Std 1003.1-2008 changed the behaivor of ranges in regular
expressions outside of the "C" and "POSIX" locales to be undefined.

Starting in 2011, gawk 4.0 stopped using the locale for the range
regular expressions and used the traditional behavior only. The
maintainer had grown weary of answering why '[A-Z]' would sometimes
match lower-case expressions. The details about are explained here:
https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html

To restore compatibility with other implementaitons of awk, revert this
patch. FreeBSD is the odd-system out. It also has the nice side effect
of eliminating the last of our differences with upstream one-true-awk.

Reviewed by: cy, rgrimes
MFC After: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31114

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Another partial revert of r301289.

In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.

Fix mismerge that crept into r301289.

The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.

Reapply r301289 by pfg:

| MFV r300961: one-true-awk: replace 0 with NULL for pointers
| Also remove a redundant semicolon.
| Also had to rebase on upstream pull.

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Fix %c for floating values that become 0 when coerced to int.

Obtained from: OpenBSD run.c 1.36 (From Jeremy Devenport)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Implement gawk multiple-arg extension to and, or, and xor.

gawk allows multiple arguemnts to bit-wiste and, or and xor
functions. Implement an arbitrary number of arguments for these
functions. Also, use NULL in preference to 0 to match rest of file.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12361

Bring in bit operation functions, ala gawk.

These are from OpenBSD:
>>> Extend awk with bitwise operations. This is an extension to the awk
>>> spec and documented as such, but comes in handy from time to time.
>>> The prototypes make it compatible with a similar GNU awk extension.
>>>
>>> ok millert@, enthusiasm from deraadt@

Edited to fix cut and paste in error messages, as well as
using tabs instead of spaces after #defines added.

Obtained From: OpenBSD awk.h 1.12, lex.c 1.10, run.c 1.29
Differential Revision: https://reviews.freebsd.org/D12361
Sponsored by: Netflix

MFV r300961:
one-true-awk: replace 0 with NULL for pointers

Also remove a redundant semicolon.

awk: Use random(3) instead of rand(3)

While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).

Use random(3) for awk as is done in other systems.

Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.

PR: 193147
MFC after: 5 weeks

MFV: one-true-awk 20121220.

MFC after: 1 month

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 7-Feb-2004 release.

Vendor import of bwk's 29-Jul-2003 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 13-Dec-2002 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Vendor import of bwk's 2002-02-18 release. Most significant update is the
inclusion of my character class patch.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

Revert "awk: Merge upstream 2nd Edition Awk Book"

The pre-push testing I did turned out to be testing the old version with
the old testsuite (for reasons I don't understnad). There's issues with
the new version, the new test in the suite or (likely) both. Revert
until they can be chased down.

This should also fix the github CI that's gone red since this commit.

This reverts commit 3fd60a6b73ac01a72df89751f173970fae4cae73, reversing
changes made to 194df014feebd8b169b41ecd75ae73d63a792d6b.

Sponsored by: Netflix

awk: Merge upstream 2nd Edition Awk Book

Merge in the November 2nd, 2023 version of one true awk.

This brings in Unicode support, CSV support and a number of bug fixes.

Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D42447

awk: errror on printf format strings lacking conversion specifier

Reported by: phk
Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39573

awk: Remove last markings we have on awk

We normally don't add $FreeBSD$ to contrib software. However, these
changes date back to the CVS era of source code management and have been
overlooked. Now that all these files are back to the same as the
upstream bsd-features branch, remove the FreeBSD specific changes, which
are now just $FreeBSD$ and the (FreeBSD) in the version string.

MFC After: 2 weeks
Sponsored by: Netflix

awk: revert to upstream behavior for ranges for gawk compatibility

In 2005, FreeBSD changed one-true-awk to honor the locale's collating
order. This was billed as a temporary patch. It was also compatible with
the then-current behavior of gawk. That temporary patch has lasted 16
years now.

However, IEEE Std 1003.1-2008 changed the behaivor of ranges in regular
expressions outside of the "C" and "POSIX" locales to be undefined.

Starting in 2011, gawk 4.0 stopped using the locale for the range
regular expressions and used the traditional behavior only. The
maintainer had grown weary of answering why '[A-Z]' would sometimes
match lower-case expressions. The details about are explained here:
https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html

To restore compatibility with other implementaitons of awk, revert this
patch. FreeBSD is the odd-system out. It also has the nice side effect
of eliminating the last of our differences with upstream one-true-awk.

Reviewed by: cy, rgrimes
MFC After: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31114

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Another partial revert of r301289.

In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.

Fix mismerge that crept into r301289.

The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.

Reapply r301289 by pfg:

| MFV r300961: one-true-awk: replace 0 with NULL for pointers
| Also remove a redundant semicolon.
| Also had to rebase on upstream pull.

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Fix %c for floating values that become 0 when coerced to int.

Obtained from: OpenBSD run.c 1.36 (From Jeremy Devenport)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Implement gawk multiple-arg extension to and, or, and xor.

gawk allows multiple arguemnts to bit-wiste and, or and xor
functions. Implement an arbitrary number of arguments for these
functions. Also, use NULL in preference to 0 to match rest of file.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12361

Bring in bit operation functions, ala gawk.

These are from OpenBSD:
>>> Extend awk with bitwise operations. This is an extension to the awk
>>> spec and documented as such, but comes in handy from time to time.
>>> The prototypes make it compatible with a similar GNU awk extension.
>>>
>>> ok millert@, enthusiasm from deraadt@

Edited to fix cut and paste in error messages, as well as
using tabs instead of spaces after #defines added.

Obtained From: OpenBSD awk.h 1.12, lex.c 1.10, run.c 1.29
Differential Revision: https://reviews.freebsd.org/D12361
Sponsored by: Netflix

MFV r300961:
one-true-awk: replace 0 with NULL for pointers

Also remove a redundant semicolon.

awk: Use random(3) instead of rand(3)

While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).

Use random(3) for awk as is done in other systems.

Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.

PR: 193147
MFC after: 5 weeks

MFV: one-true-awk 20121220.

MFC after: 1 month

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 7-Feb-2004 release.

Vendor import of bwk's 29-Jul-2003 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 13-Dec-2002 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Vendor import of bwk's 2002-02-18 release. Most significant update is the
inclusion of my character class patch.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

awk: errror on printf format strings lacking conversion specifier

Reported by: phk
Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39573

awk: Remove last markings we have on awk

We normally don't add $FreeBSD$ to contrib software. However, these
changes date back to the CVS era of source code management and have been
overlooked. Now that all these files are back to the same as the
upstream bsd-features branch, remove the FreeBSD specific changes, which
are now just $FreeBSD$ and the (FreeBSD) in the version string.

MFC After: 2 weeks
Sponsored by: Netflix

awk: revert to upstream behavior for ranges for gawk compatibility

In 2005, FreeBSD changed one-true-awk to honor the locale's collating
order. This was billed as a temporary patch. It was also compatible with
the then-current behavior of gawk. That temporary patch has lasted 16
years now.

However, IEEE Std 1003.1-2008 changed the behaivor of ranges in regular
expressions outside of the "C" and "POSIX" locales to be undefined.

Starting in 2011, gawk 4.0 stopped using the locale for the range
regular expressions and used the traditional behavior only. The
maintainer had grown weary of answering why '[A-Z]' would sometimes
match lower-case expressions. The details about are explained here:
https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html

To restore compatibility with other implementaitons of awk, revert this
patch. FreeBSD is the odd-system out. It also has the nice side effect
of eliminating the last of our differences with upstream one-true-awk.

Reviewed by: cy, rgrimes
MFC After: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31114

one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs

Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by: Mikolaj Golub <to.my.trociny@gmail.com>
PR: 143363,143365,143368,143369,143373,143375,214782
Sponsored by: Netflix

Another partial revert of r301289.

In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.

Fix mismerge that crept into r301289.

The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.

Reapply r301289 by pfg:

| MFV r300961: one-true-awk: replace 0 with NULL for pointers
| Also remove a redundant semicolon.
| Also had to rebase on upstream pull.

Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git

Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).

Fix %c for floating values that become 0 when coerced to int.

Obtained from: OpenBSD run.c 1.36 (From Jeremy Devenport)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Implement gawk multiple-arg extension to and, or, and xor.

gawk allows multiple arguemnts to bit-wiste and, or and xor
functions. Implement an arbitrary number of arguments for these
functions. Also, use NULL in preference to 0 to match rest of file.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12361

Bring in bit operation functions, ala gawk.

These are from OpenBSD:
>>> Extend awk with bitwise operations. This is an extension to the awk
>>> spec and documented as such, but comes in handy from time to time.
>>> The prototypes make it compatible with a similar GNU awk extension.
>>>
>>> ok millert@, enthusiasm from deraadt@

Edited to fix cut and paste in error messages, as well as
using tabs instead of spaces after #defines added.

Obtained From: OpenBSD awk.h 1.12, lex.c 1.10, run.c 1.29
Differential Revision: https://reviews.freebsd.org/D12361
Sponsored by: Netflix

MFV r300961:
one-true-awk: replace 0 with NULL for pointers

Also remove a redundant semicolon.

awk: Use random(3) instead of rand(3)

While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).

Use random(3) for awk as is done in other systems.

Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.

PR: 193147
MFC after: 5 weeks

MFV: one-true-awk 20121220.

MFC after: 1 month

Update to a 7-Aug-2011 release.

Approved by: re (kib)

Update to a 1-May-2011 release (except for the isblank change).

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

Update to a 26-Nov-2009 release.

Flatten out vendor tree.

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 7-Feb-2004 release.

Vendor import of bwk's 29-Jul-2003 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 13-Dec-2002 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Vendor import of bwk's 2002-02-18 release. Most significant update is the
inclusion of my character class patch.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.

Another partial revert of r301289.

In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.

Fix mismerge that crept into r301289.

The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.

Reapply r301289 by pfg:

| MFV r300961: one-true-awk: replace 0 with NULL for pointers
| Also remove a redundant semicolon.
| Also had to rebase on upstream pull.

Fix %c for floating values that become 0 when coerced to int.

Obtained from: OpenBSD run.c 1.36 (From Jeremy Devenport)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12379

Implement gawk multiple-arg extension to and, or, and xor.

gawk allows multiple arguemnts to bit-wiste and, or and xor
functions. Implement an arbitrary number of arguments for these
functions. Also, use NULL in preference to 0 to match rest of file.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12361

Bring in bit operation functions, ala gawk.

These are from OpenBSD:
>>> Extend awk with bitwise operations. This is an extension to the awk
>>> spec and documented as such, but comes in handy from time to time.
>>> The prototypes make it compatible with a similar GNU awk extension.
>>>
>>> ok millert@, enthusiasm from deraadt@

Edited to fix cut and paste in error messages, as well as
using tabs instead of spaces after #defines added.

Obtained From: OpenBSD awk.h 1.12, lex.c 1.10, run.c 1.29
Differential Revision: https://reviews.freebsd.org/D12361
Sponsored by: Netflix

awk: Use random(3) instead of rand(3)

While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).

Use random(3) for awk as is done in other systems.

Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.

PR: 193147
MFC after: 5 weeks

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Update to a 26-Nov-2009 release.

Apply patches directly to sources. Their effect is as follows:

- Make one-true-awk respect locale's collating order in [a-z]
bracket expressions, until a more complete fix (like handing
BREs) is ready.

- Don't require a space between -[fv] and its argument.

- Import the HEAD csup code which is the basis for the cvsmode work.

Vendor import of bwk's 01-May-2007 release.

Approved by: delphij (mentor)
Nodded by: ru
Tested by: make universe

Vendor import of bwk's 24-Apr-2005 release.

Vendor import of bwk's 7-Feb-2004 release.

Vendor import of bwk's 29-Jul-2003 release.

Vendor import of bwk's 14-March-2003 release.

Vendor import of bwk's 13-Dec-2002 release.

Vendor import of bwk's 29-Nov-2002 release.
Most significant update is the inclusion of our port's locale patches.

Vendor import of bwk's 2002-02-18 release. Most significant update is the
inclusion of my character class patch.

Import the One True AWK, in the form of bwk's AWK Nov 15, 2000 release.