Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

sort: replace malloc+memset with calloc

sort: Cache value of MB_CUR_MAX

Every usage of MB_CUR_MAX results in a call to __mb_cur_max. This is
inefficient and redundant. Caching the value of MB_CUR_MAX in a global
variable removes these calls and speeds up the runtime of sort. For
numeric sorting, runtime is almost halved in some tests.

PR: 255551
PR: 255840
Reviewed by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30170

sort(1): Fix two wchar-related bugs in radixsort

Sort(1)'s radixsort implementation was broken for multibyte LC_CTYPEs in at
least two ways:

* In actual radix sort, it would only bucket the least significant
byte from each wchar, ignoring the 24 most-significant bits of each
unicode character.

* In degenerate cases / "fast paths," it would fall back to another
sorting algorithm (default: mergesort) with a bogus comparator
offset. The string comparison functions in sort(1) take an offset
in units of the operating character size. However, radixsort was
passing an offset in units of bytes. The byte offset must be
divided by sizeof(wchar_t).

This revision addresses both discovered issues.

Some example testcases:

$ (echo 耳 ; echo 脳 ; echo 耳) | \
LC_CTYPE=ja_JP.UTF-8 LC_COLLATE=C LANG=C sort --radixsort --debug

$ (echo 耳 ; echo 脳 ; echo 耳) | \
LC_CTYPE=C LC_COLLATE=C LANG=C sort --radixsort --debug

$ (for i in $(jot 34); do echo 耳耳耳耳耳; echo 耳耳耳耳脳; echo 耳耳耳耳脴; done) | \
LC_CTYPE=ja_JP.UTF-8 LC_COLLATE=C LANG=C sort --radixsort --debug

PR: 247494
Reported by: knu
MFC after: I do not intend to, but parties interested in stable might want to

Fix the WITH_SORT_THREADS build.

PR: 201664
MFC after: 1 week

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

sort - Don't live-loop threads.

Worker threads now use a pthread_cond_t to wait for work instead of
burning the cpu up.

Obtained from: DragonflyBSD (07774aea0ccf64a48fcfad8899e3bf7c8f18277a)
MFC after: 2 weeks

- Use correct offsets into the keys set array. As the elements of this
zero-length array are dynamically sized at run-time based on the use
of hints, compilers can't be expected to figure out these offsets on
their own. [1]
- Fix incorrect comparison in cmp_nans(). [2]

PR: 204571 [1], 202301 [2]
Submitted by: David Binderman [2]
MFC after: 3 days

sort: style knits / cleanups.

Minor cleanups that got accidentally reverted.

Obtained from: OpenBSD

Revert (partial) r281123, r281125:
sort: style knits / cleanups.

Our style guide(9) specifies that in absence of local variables
an empty line must be inserted.

Pointed out by: eadler

sort: style knits / cleanups.

Obtained from: OpenBSD

sort: Cleanup small issues with spaces.

Obtained from: OpenBSD

In usr.bin/sort/radixsort.c, pop_ls_mt() is only referenced if
SORT_THREADS is defined, so make the whole function conditional, instead
of just the pthread calls in it.

MFC after: 3 days

- Update Oleg Moskalenko's email address

Requested by: Oleg Moskalenko <mom040267@gmail.com>

- Portability changes for ARM
- Allow larger sort memory on 64-bit platforms

Submitted by: Oleg Moskalenko <oleg.moskalenko@citrix.com>

- Change --nthreads parameter to --parallel for GNU compatibility
- Change default sort method to mergesort, which has a better worst case
performance than qsort

Submitted by: Oleg Moskalenko <oleg.moskalenko@citrix.com>

- Only use multi-threading for large files
- Do not use mmap() by default; it can be enabled by --mmap
- Add some minor optimizations for -u
- Update manual page according to the changes

Submitted by: Oleg Moskalenko <oleg.moskalenko@citrix.com>

- Eliminate initializations if global variables. Compilers are not
required to optimize these so it may result in larger binary size.

Pointed out by: kib

Add a BSD-licensed sort rewrite that was started by me and later completed
with the major functionality and optimizations by Oleg Moskalenko.
It is compatible with the latest version of POSIX and the current GNU sort
version that we have in base. Beside this, it implements all the
functionality introduced in later versions of GNU sort. For now, it will
be installed as "bsdsort", keeping GNU sort as the default sort
implementation.