Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

pkgbase: Put more binaries/lib in runtime

Move some needed binaries/libs from FreeBSD-utilities to FreeBSD_runtime.
This is everything needed to boot to multiuser with FreeBSD-rc installed.

MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33435

grep: fix LINKS in Makefile

zstdegrep was listed twice, instead of zstdfgrep

Pull Request: https://github.com/freebsd/freebsd-src/pull/450

build: remove the option to build gnugrep

Unconditionally install bsdgrep as grep, bootstrap or not. Remove all
build glue and stop installing both gnugrep and libgnuregex now that
all consumers of the latter are gone.

Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D27732

bsdgrep: don't link against libregex for bootstrap

r368355 removed the GNU_GREP_COMPAT knob (off by default) and forgot that
bsdgrep may be built/used for bootstrap on some systems.

All base uses should strive to use only POSIX-compliant expressions anyways
and we haven't had libregex by default here up to this point, so just don't
do that if we're bootstrapping.

Note that the resulting binary has the wrong `grep -V` information as it
falsely claims to be GNU compatible, but it is only for bootstrap.

Reported by: GitHub cross-builds via yuripv

Retire GNU_GREP_COMPAT knob

This was introduced and then disabled by default primarily to avoid dealing
with bugs in libgnuregex. rS363823 switched to using libregex for it, so
let's just rip the option out now so we can make sure we're getting tested
with libregex via bsdgrep.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D27476

Always install usr.bin/grep as grep when bootstrapping

We have to bootstrap grep when cross-building from macOS/Linux.

bsdgrep: switch to libregex for GNU_GREP_COMPAT

libregex is incomplete, but it's a bit less buggy than the in-base
libgnuregex and mostly OK.

While here, rename -DIWTH_GNU -> -DWITH_GNU_COMPAT; the option implies
that we're compatible with the GNU counterpart, not that we're including GNU
anything.

Fix bsdgrep manpage clobbering grep(1) with default build options

The default build should install bsdgrep(1) and grep(1), with the latter
being gnugrep(1). WITH_BSD_GREP flips this situation such that we have
gnugrep(1) and grep(1), with the latter being bsdgrep(1).

Changes to start installing the zgrep script out of usr.bin/grep
inadvertently altered the default build such that grep(1) was being
installed, and it was bsdgrep(1). Correct the typo.

Reported by: bcran
MFC after: 3 days

Remove NLS support from BSD grep

GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.

We do support 193 locales in base, we will never catch up on that number of
translation with bsd grep.

Removing NLS support make bsd grep consistent with the other binaries in base
which are not translated, and also reduce a little bit the code.

Reviewed by: kevans
Approved by: kevans
Discussed with: kevans @BSDCan
Differential Revision: https://reviews.freebsd.org/D15682

bsdgrep: annihilate our in-tree TRE, previously disabled by default

It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).

To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.

Remove compression support from bsdgrep

Compression support is now handled by an external script, remove it from the
bsdgrep(1) utility.
This removes the support for -Z -J -X and -M

Note: that it matches the changes in newer GNU grep

Reviewed by: kevans
Approved by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15197

Use a script wrapper for <compress>grep

Import the wrapper script from zstdgrep (written by wiz@netbsd.org)

Modify it to support more than just zstd (adding support for gzip,
lzma, xz and bzip2)

Write a simple manpage dedicated for it.

Only use that new wrapper both for gnu grep and bsd grep

Next step will be removing code related to compression format from bsdgrep

Reviewed by: kevans
Approved by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15193

Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.

Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after: 1 weeks

Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.

Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
A LIBSTAND hack is no longer required for buildenv.

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

bsdgrep: Add GNU compatible version string indicator

As reported in r218614 it's useful to have an indication of whether or not
BSD grep was built with GNU_GREP_COMPAT.

Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reported by: mandree
Differential Revision: https://reviews.freebsd.org/D10451

bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatch

Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review
(D10098).

In comparison with the fastmatch implementation, Kyle Evans found that:

- regex(3)'s performance with literal expressions offers a speed
improvement over fastmatch

- regex(3)'s performance, both with simple BREs and EREs, seems to be
comparable

The regex implementation was imported in r226035, and the commit message
reports:

This is a temporary solution until the whole regex library is
not replaced so that BSD grep development can continue and the
backported code gets some review and testing. This change only
improves scalability slightly, there is no big performance boost
yet but several minor bugs have been found and fixed.

Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.

PR: 175314, 194823
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: bdrewery (in part)
Differential Revision: https://reviews.freebsd.org/D10282

bsdgrep(1): Rip out "xmalloc" bits

xmalloc was a debug malloc implementation, but the x{malloc,calloc,free}
functions default to calling the malloc(3) equivalents.

Instead of relying on this malloc shim, we can devise better ways to debug
malloc issues that aren't misleading upon initial inspection. (I.e., using
jemalloc's various built-in debugging capabilities.)

Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: emaste, cem
Differential Revision: https://reviews.freebsd.org/D10269

bsdgrep(1): create rgrep link

Create a convenience rgrep link for bsdgrep(1) that observes 'grep -r'
behavior.

A follow-up to r316473.

Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: emaste (earlier version), cem
Differential Revision: https://reviews.freebsd.org/D10109

Bump GCC max-inline-insns-single in libiconv_modules and grep

This is required by our FORTIFY_SOURCE implementation as it
does more inlining. As a rule of thumb, FORTIFY_SOURCE doubles
the number of inlines except that in grep inlining
blows up for some reason.

Avoid bogus MLINKS when MK_LZMA_SUPPORT=yes and MK_BSD_GREP=no

Create MANLINKS for lzgrep, etc when MK_LZMA_SUPPORT != no

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Add a ${CP} alias for copying files in the build.

Some users build FreeBSD as non-root in Perforce workspaces. By default,
Perforce sets files read-only unless they're explicitly being edited.
As a result, the -f argument must be used to cp in order to override the
read-only flag when copying source files to object directories. Bare use of
'cp' should be avoided in the future.

Update all current users of 'cp' in the src tree.

Reviewed by: emaste
MFC after: 1 week
Sponsored by: Spectra Logic

Convert to usr.bin/ to LIBADD
Reduce overlinking

Integrate usr.bin/grep/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Check in first src/tests snapshot from NetBSD anoncvs

Sources were obtained like so:

% export CVSROOT="anoncvs@anoncvs.NetBSD.org:/cvsroot"
% cvs -z9 co -D "09/30/2014 20:45" -P src/tests
% mv src/tests/* tests/dist/.

'*CVS*' has been added to svn:ignore to ease updating periodically from
upstream

Some line ending issues had to be resolved with test outputs and scripts
via dos2unix and by deleting the eol-style property set in usr.bin/sort

Discussed with: rpaulo
Sponsored by: EMC / Isilon Storage Division

Fix incremental builds involving non-root users with read-only source files.

Makefiles should not assume that source files can be overwritten. This is the
common case for Perforce source trees.

This is a followup commit to r211243 in the same vein.

MFC after: 1 month
Sponsored by: Spectra Logic
MFSpectraBSD: r1036319 on 2014/01/29, r1046711 on 2014/03/06

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Don't test WITHOUT_FOO in program makefiles, test MK_FOO instead.

Adjust CFLAGS to pick up correct regex.h and posix/regex.h. Note this
actually reverts r250860 and r250861.

Reported by: gjb, tinderbox

Following r226271, allow disabling lzma support with "WITHOUT_LZMA_SUPPORT".
Correct r226271 which should have used WITHOUT_BZIP2_SUPPORT per r166255.

Obtained from: Juniper Networks

- Create links to the xz and lzma versions even if BSD grep is not the
default. Nor GNU nor liblzma in base provides such functionality so
it may be useful.

MFC after: 3 days

- Fix installation when WITH_BSD_GREP is set to yes

Submitted by: Aleksandr Rybalko <ray@ddteam.net>

- Use getprogname() instead of __progname
- Allow disabling bzip2 support with WITHOUT_BZIP2
- Fix handling patterns that start with a dot
- Remove superfluous semicolon

Approved by: delphij (mentor)

Update BSD grep to the latest development version. It has some code
backported that was written for the TRE integration project in Google
Summer of Code 2011. This is a temporary solution until the whole
regex library is not replaced so that BSD grep development can continue
and the backported code gets some review and testing. This change only
improves scalability slightly, there is no big performance boost yet
but several minor bugs have been found and fixed.

Approved by: delphij (mentor)
Sposored by: Google Summer of Code 2011
MFC after: 1 week

Remove redundant assignments to WARNS.

For these directories, WARNS is already implied to be 6.

Build and install a BSD licensed grep.
If WITH_BSD_GREP is not set, it will be 'bsdgrep' and GNUgrep will be
'[ef]grep'. Otherwise, BSD-grep will be the grep family, and GNUgrep
will be 'gnugrep'.

Discussed with: brooks

- Adjust a comment to actual behaviour
- Makefile nit
- Add more CVS/SVN keywords to make it easier to track changes from NetBSD
in case they add further improvements

Approved by: delphij (mentor)
Obtained from: The NetBSD Project

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Add BSD grep to the base system and make it our default grep.

Deliverables: Small and clean code (1,4 KSLOC vs GNU's 8,5 KSLOC),
lower memory usage than GNU grep, GNU compatibility,
BSD license.

TODO: Performance is somewhat behind GNU grep but it is only
significant for bigger searches. The reason is complex, the
most important factor is that GNU grep uses lots of
optimizations to improve the speed of the regex library.
First, we need a modern regex library (practically by adopting
TRE), add support for GNU-style non-standard regexes and then
reevalute the performance issues and look for bottlenecks. In
the meantime, for those, who need better performance, it is
possible to build GNU grep by setting WITH_GNU_GREP.

Approved by: delphij (mentor)
Obtained from: OpenBSD (http://www.openbsd.org/cgi-bin/cvsweb/src/usr.bin/grep/),
freegrep (http://github.com/howardjp/freegrep)
Sponsored by: Google SoC 2008
Portbuild tests run by: kris, pav, erwin
Acknowledgements to: fjoe (as SoC 2008 mentor),
everyone who helped in reviewing and testing