loader: add target for dirdeps build

Update dependencies for the loader variations used for each
architecture.

Reviewed by: sjg
Differential Revision: https://reviews.freebsd.org/D39741

Add support for host32 for DIRDEPS_BUILD

Allow building 32bit libs for host.

Move CFLAGS additions from local.sys.dirdeps.mk (which is too early
and impacts CFLAGS defaults) to local.sys.mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D41946

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

local.dirdeps.mk ensure tools/build is built if needed

If MK_host_egacy is yes, we need it built, make sure that happens.

Reviewed by: stevek

local.dirdeps.mk no qual for PSEUDO_MACHINE_LIST

dirdeps.mk takes care of "host"
but we still need to deal with other PSEUDO_MACHINE_LIST members

local.dirdeps.mk skip N_host_libs for non-FreeBSD host

The N_host_libs dance only makes sense if host is FreeBSD.
Even then, if MK_host_egacy is yes we need libmd

libnetbsd does not need libutil when using libegacy

Sponsored by: Juniper Networks, Inc.

Simplify building host tools during DIRDEPS_BUILD

The whole point of the DIRDEPS_BUILD is to avoid tree walks
and basically build everything in a single pass.
We use the pseudo MACHINE "host" to represent the build host.

When the build host is not FreeBSD or is an older version of FreeBSD
it may need some help to build host-tools.

The directory tools/build does this - building libegacy.

local.sys.mk: create a pseudo option MK_host_egacy to indicate
if tools/build needs to be built for "host".

local.dirdeps.mk: set MK_host_egacy.host to ${MK_host_egacy}
all other DEP_MACHINES will get "no"

This allows a Makefile.depend.options in makefs etc to cause tools/build
to be built for host but only if necessary.

local.init.mk: use ISYSTEM as arg to -isystem so that it can be overridden.
The default remains ${STAGE_INCLUDEDIR}

src.init.mk: if MACHINE is host and we are not FreeBSD
set some MK_ flags the same as tools/build/mk/Makefile.boot.pre and
include src.init.${.MAKE.OS:tl}.mk if it exists.

For older versions of FreeBSD add libegacy when building PROGs for "host"

Also instead of -isystem${STAGE_INCLUDEDIR} we want
-I${STAGE_INCLUDEDIR} and -isystem/usr/include so we override ISYSTEM.
This means any headers we stage for "host" will take precedence over
system headers but #include_next will DTRT.

src.init.linux.mk: add
-I${SRCTOP}/tools/build/cross-build/include/linux
and generally deal with building host tools on Linux.
Eg. static linking does not work so set NO_SHARED= no
Override some HAVE_ flags.

src.sys.env.mk: on linux awk throws an warning about # in newvers.sh
just send stderr to /dev/null

Reviewed by: jrtc27, arichardson
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39744

Update meta.sys.mk and related local*mk

Move the setting of TARGET_SPEC_VARS to local.sys.env.mk
so meta.sys.mk can do the processing, and include local.meta.sys.mk
later.

Move the setting of GENDIRDEPS_FILTER*_VARS from local.gendirdeps.mk
to local.meta.sys.mk so we can automatically set DEP_* at level 1+
to avoid syntax errors when DEP_* variables are used in conditionals
in Makefile.depend files.

Update gendirdeps.mk just to get the documentation about the above.
No functional change.

local.dirdeps.mk be more careful about adding to DIRDEPS to avoid
unnecessary overhead, and introducing cycles in the graph.
Also set DEP_MACHINE_CPUARCH.

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39737

Allow site.*.mk to augment local.*.mk and src.*.mk

Add some extra customization points so that FreeBSD build
can be adapted to local requirements.
We use these to minimize changes to share/mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37617

Update dirdeps.mk et al to latest

Move some local tweaks to local.*.mk

Reviewed by: bdrewery
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27300

retire the LLVM_LIBUNWIND option

LLVM's libunwind is used on all FreeBSD-supported CPU architectures and
is a required component.

Reviewed by: brooks (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23123

remove GCC 4.2.1 build infrastructure

As described in Warner's email message[1] to the FreeBSD-arch mailing
list we have reached GCC 4.2.1's retirement date. At this time all
supported architectures either use in-tree Clang, or rely on external
toolchain (i.e., a contemporary GCC version from ports).

GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later
that year, in r171825. GCC has served us well, but version 4.2.1 is
obsolete and not used by default on any architecture in FreeBSD. It
does not support modern C and does not support arm64 or RISC-V.

Thanks to everyone responsible for maintaining, updating, and testing
GCC in the FreeBSD base system over the years.

So long, and thanks for all the fish.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR: 228919
Reviewed by: brooks, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23124

retire BSD_CRTBEGIN option

BSD crt is currently used on all architectures (other than sparc64).
Remove the option and use BSD crt everywhere as part of the GCC 4.2.1
retirement plan.

https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR: 239851
Reviewed by: andrew, brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23122

Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943

Tweaks for DIRDEPS_BUILD

libmagic only depend on mkmagic if not DIRDEPS_BUILD

libpmc fix -I for libpmcstat

local.dirdeps.mk be even more careful about adding gnu/lib/csu to DIRDEPS

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D22872

Update dirdeps.mk and gendirdeps.mk

The env space consumed by exporting all libc's .meta files
left little room for command line,
so unexport when done.

Update dirdeps.mk to latest and add
dirdeps-targets.mk to simplify/update targets/Makefile

Makefile changes to go with Makefile.depend changes in D22494

Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22495

Update dirdeps.mk et al to latest

dirdeps.mk and meta.autodep.mk will now look for
Makefile.depend.options
to handle optional dependencies, the work is all done by
dirdeps-options.mk

Also update to latest meta.stage.mk and gendirdeps.mk

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D15701

DIRDEPS_BUILD: Cleanup leftover WORLDTMP files.

This will force any existing objects to rebuild if their .meta
files reference files from WORLDTMP. This is a problem after
the UNIFIED_OBJDIR effort caused buildworld and DIRDEPS_BUILD
to share an OBJDIR. Without cleaning these files the
Makefile.depend files end up with odd tmp/legacy/... entries
since some bootstrap-tools linger from there and otherwise
don't rebuild. Removing the files causes anything having
used WORLDTMP to rebuild.

Sponsored by: Dell EMC Isilon

DIRDEPS_BUILD: libgcc now depends on MK_LLVM_LIBUNWIND

The dependency on gnu/lib/libgcc or lib/libgcc* is determined
at 'make dirdeps' time.

Sponsored by: Dell EMC Isilon

DIRDEPS_BUILD: Exclude secure/lib* libraries for host builds.

Sponsored by: EMC / Isilon Storage Division

Remove redundant code imported into dirdeps.mk in r290956.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Let PROGS bootstrapping work.

- Support (DP|LIB)ADD_${PROG}.
- Support SRCS[._]${PROG}.
- Don't bootstrap DIRDEPS while recursing.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: For bootstrapping always install all headers.

There is no good way to guess if any of these will be needed but
they commonly are and add no extra overhead so just stage them.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Support bootstrapping from PROGS.(DPADD|LIBADD).

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Fix incorrectly adding in RELDIR for DIRDEPS in bootstrapping.

This is not wrong, but was unexpected. Using <empty>:H results in '.' which
then using the rest of the conversion was added in RELDIR. This was also
causing an empty _DP_DIRDEPS to resolve to SRCTOP for DIRDEPS.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Avoid duplicates in DIRDEPS for bootstrapping.

This logic is potentially included multiple times, so overwrite the temporary
variable rather than append to it.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Don't force C dependencies on kernel modules for bootstrapping.

These use ld(1), effectively -nostdlib, and don't need any of these
normal dependencies.

kmod builds also define PROG so just checking for KMOD here seems to be
the easiest to handle it.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: For the bootstrapped LIBADD from DPADD, resolve paths to RELDIR.

This allows the LIBDEPS/DPADD for the clang build to not have
../../../lib/clang/* in DIRDEPS.

Sponsored by: EMC / Isilon Storage Division

META MODE: Add some basic bootstrapping support for no Makefile.depend.

This will not work for bootstrapping dependencies, it will only
bootstrap the top-level build directory.

Sponsored by: EMC / Isilon Storage Division

META MODE: Don't trim out lib/clang/include dependency.

Doing this causes more trouble than it is worth regarding cyclic
dependencies. It should not be needed after cleaning up MACHINE=host
builds in r291324.

Sponsored by: EMC / Isilon Storage Division

Standardize on OBJTOP in and outside of META MODE.

Sponsored by: EMC / Isilon Storage Division

META MODE: Fix 'make bootstrap-tools'.

The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.

This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.

For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds. This has not yet been a problem due
to readelf not being built as a host tool in buildworld. This is possible
in the meta build though when building the toolchain.

Sponsored by: EMC / Isilon Storage Division

Watch out for bin/cat in host stage tree

Move META_MODE bits to local.meta.sys.mk

By moving META_MODE bits from local.sys.mk, they are easier
to skip when MK_META_MODE=no

Update some filters to cope with sync from head.

If buildworld etc or WITHOUT_META_MODE disable all the META_MODE
related options.

Renamed pkgs/ targets/

There are a few libs we need to build for host.
Handle more optional dependencies.

We now want src.opts.mk

Sadly there is at least on lib we need to build for host (libdwarf)
else we cannot build ctfconvert.

Leverage bsd.opts.mk

Some re-org and cleanup.
Make it easier to subst some dirs in dirdeps with variables.
Tweak M_dep_qual_fixes esp. for pseudo machines like "host"
so that it can DTRT against _machines (in dirdeps.mk).

Keep dirdeps for pseudo machines like "host" and "common" simple.
Also we need to use DEP_MACHINE_ARCH when expanding CSU_DIR

Don't add libssp for host

Improve DIRDEP filtering.
Allow DEP_SKIP_DIR and DEP_DIRDEPS_FILTER to vary by DEP_MACHINE.

A bit finer control so we can build toolchain for host

Apply DEP_DIRDEPS_FILTER.${DEP_MACHINE} as needed.
Use DEP_DIRDEPS_FILTER.host to supress dependencies for host tools.

Use pkgs/pseudo/stage to run mtree against STAGE_OBJTOP

Sync FreeBSD's bmake branch with Juniper's internal bmake branch.

Requested by: Simon Gerraty <sjg@juniper.net>