Guard CROSS_TARGET_FLAGS from buildworld

Until a better arrangment is worked out, guard the setting of
CROSS_TARGET_FLAGS in local.sys.mk with check for DIRDEPS_BUILD.

Using a separate flag for CROSS_TARGET that can be reset by
bsd.compat.mk is probably the right thing.

Add support for host32 for DIRDEPS_BUILD

Allow building 32bit libs for host.

Move CFLAGS additions from local.sys.dirdeps.mk (which is too early
and impacts CFLAGS defaults) to local.sys.mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D41946

Update jobs.mk to leverage -jC

With bmake-20230909 we can use -j1.5C to use 1.5*ncpu
bmake will set .MAKE.JOBS.C=yes if this is supported,
and in that case jobs.mk will use JOB_MAX_C as default for JOB_MAX.

Remove logic from local.sys.mk which is now handled by jobs.mk

Also use latest dirdeps-targets.mk and meta.{autodep,subdir}.mk

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Set MK_host_egacy=yes earlier

If we need to set MK_host_egacy=yes we might need to check it
in local.toolchain.mk so set it earlier in local.meta.sys.env.mk

Leave the default MK_host_egacy?=no in local.sys.mk so it can
be tested in Makefiles without concern for build mode.

Ensure good exit status from type

When looking to see if nproc is available we do
not want warnings about 'type nproc' having bad exit.

Simplify building host tools during DIRDEPS_BUILD

The whole point of the DIRDEPS_BUILD is to avoid tree walks
and basically build everything in a single pass.
We use the pseudo MACHINE "host" to represent the build host.

When the build host is not FreeBSD or is an older version of FreeBSD
it may need some help to build host-tools.

The directory tools/build does this - building libegacy.

local.sys.mk: create a pseudo option MK_host_egacy to indicate
if tools/build needs to be built for "host".

local.dirdeps.mk: set MK_host_egacy.host to ${MK_host_egacy}
all other DEP_MACHINES will get "no"

This allows a Makefile.depend.options in makefs etc to cause tools/build
to be built for host but only if necessary.

local.init.mk: use ISYSTEM as arg to -isystem so that it can be overridden.
The default remains ${STAGE_INCLUDEDIR}

src.init.mk: if MACHINE is host and we are not FreeBSD
set some MK_ flags the same as tools/build/mk/Makefile.boot.pre and
include src.init.${.MAKE.OS:tl}.mk if it exists.

For older versions of FreeBSD add libegacy when building PROGs for "host"

Also instead of -isystem${STAGE_INCLUDEDIR} we want
-I${STAGE_INCLUDEDIR} and -isystem/usr/include so we override ISYSTEM.
This means any headers we stage for "host" will take precedence over
system headers but #include_next will DTRT.

src.init.linux.mk: add
-I${SRCTOP}/tools/build/cross-build/include/linux
and generally deal with building host tools on Linux.
Eg. static linking does not work so set NO_SHARED= no
Override some HAVE_ flags.

src.sys.env.mk: on linux awk throws an warning about # in newvers.sh
just send stderr to /dev/null

Reviewed by: jrtc27, arichardson
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D39744

Add jobs.mk to allow for target-jobs

jobs.mk automates -j$JOB_MAX and capturing build log based on target.

Compute a default for JOB_MAX in local.sys.mk

Reviewed by: stevek, imp
Differential Revision: https://reviews.freebsd.org/D39683

meta: print some additional variables on build errors.

Add CPUTYPE and MACHINE_CPUARCH to the list of variables printed
when there are build errors.

Protect local.sys.mk from multiple inclusion.

Reviewed by: sjg
Obtained from: Juniper Networks, Inc.

Allow site.*.mk to augment local.*.mk and src.*.mk

Add some extra customization points so that FreeBSD build
can be adapted to local requirements.
We use these to minimize changes to share/mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37617

META_MODE: Delete build targets that fail.

If a meta mode change is triggered but then the build fails then the
next build will not retrigger meta mode. This only prevented by
removing the target on rebuild or on the failure to rebuild.

Sponsored by: DellEMC

Follow-up r320061: Need to respect make.conf/env LIBDIR overrides.

This fixes the lib32 build from creating all stale .depend files.

X-MFC-With: 320061
MFC after: 1 week
Sponsored by: Dell EMC Isilon

WITH_META_MODE: End each ERROR_CMD CMD line with ';'.

This makes it easier to debug multi-line command failures.

X-MFC-With: r319862
MFC after: 1 week
Sponsored by: Dell EMC Isilon

WITH_META_MODE: Don't try showing command if .ERROR_META_FILE is empty.

This was sed'ing on stdin for failing .PHONY targets.

Reported by: Mark Millard
X-MFC-With: r319862
MFC after: 1 week
Sponsored by: Dell EMC Isilon

META_MODE: Show .ERROR_CMD in error.

This uses a hack to get the CMD from the meta file rather than
.ERROR_CMD since bmake currently blanks the value for non-jobs
mode.

Reviewed by: sjg (indirectly)
MFC after: 1 week
Sponsored by: Dell EMC Isilon

WITH_META_MODE: Enable printing of some of make's environment on error.

This will print a set of variables from make on error using
MAKE_PRINT_VAR_ON_ERROR. It is already enabled for the DIRDEPS_BUILD.
It may make sense to enable this in the non-meta mode as well once
people are more used to its more verbose error output.

This makes it much simpler to see which .meta file is used when a
command files so that it may be inspected for the build command.

Suggested by: sjg
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

Follow-up r297835: Let the intented default cookie work.

This happened to work for not prepending .OBJDIR twice but broke the
other case of prepending it when needed.

Pointyhat to: bdrewery
Sponsored by: EMC / Isilon Storage Division

Rework META_TARGETS so that it automatically adds META_DEPS to the targets.

This will only be done if the target is defined, so if the target is
defined after bsd.sys.mk is included then it needs to manually add
${META_DEPS} still.

Sponsored by: EMC / Isilon Storage Division

META_MODE: Support targets that already have .OBJDIR in them for META_COOKIE.

Sponsored by: EMC / Isilon Storage Division

Define OBJTOP earlier if possible for use in targets.

Sponsored by: EMC / Isilon Storage Division

Follow-up r296700: Fix incorrectly declaring these as .MAIN.

This broke bootstrap-tools at least for DIRDEPS_BUILD.

Sponsored by: EMC / Isilon Storage Division

META_MODE: We can only use a cookie if filemon is being used.

Sponsored by: EMC / Isilon Storage Divsion

META_MODE: Simplify the META_COOKIE handling to use .USE/.USEBEFORE.

Extend it to other cases of meta mode cookies so they get the proper rm
cookie behavior when a .meta file detects it needs to rebuild and fails.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Remove the cookie when target is out-of-date.

The meta file may decide the target is out of date but nothing
ensures that the *next* build will build this target if it
fails this time for some reason; it is still out-of-date
until it succeeds.

Convert the include/ cookie usage to the global versions.

Sponsored by: EMC / Isilon Storage Division

Define .MAKE.MODE to normal to avoid the need for :U later.

Sponsored by: EMC / Isilon Storage Division

META MODE: Show PATH on errors.

Sponsored by: EMC / Isilon Storage Division

Rename META_MODE option to DIRDEPS_BUILD

This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.

Differential Revision: https://reviews.freebsd.org/D4153
Reviewed by: bdrewery

Move include of make.conf back to its old position.

This means moving include of local.sys.mk and src.sys.mk too.
Introduce new includes to take the early slot, for the purpose
of being able to influence toolchains and the like.

Differential Revision: D2860
Reviewed by: imp

Skip MAKE_PRINT_VAR_ON_ERROR unless we are doing META_MODE
with all the recursion in normal build it is too much noise.

Allow AUTO_OBJ with out META_MODE.
Remove frobbing of SRCCONF etc - not strictly relevant to meta mode.

Move META_MODE bits to local.meta.sys.mk

By moving META_MODE bits from local.sys.mk, they are easier
to skip when MK_META_MODE=no

Update some filters to cope with sync from head.

If buildworld etc or WITHOUT_META_MODE disable all the META_MODE
related options.

Move include of auto.obj.mk to sys.mk

Crude cross-build support thanks to clang

export sanitized MAKESYSPATH in case .../ appears in default syspath.

Use _LIBDIR in STAGE_LIBDIR so we DTRT for PRIVATE libs.
We now use SYSROOT
and (for now) non-shared toolchain when building for host.

When bootstrapping tools MACHINE==host and EARLY_BUILD defined,
so not use any of stage tree.

M_whence only first value counts

Avoid /etc/make.conf
and make it easier to deal with compiler updates

Handle the different expectations of MACHINE_ARCH b/w buildworld etc
and dirdeps.

Some cleanup and re-org.
Also update CFLAGS_LAST.clang to match new compiler version.
Add BOOT_MACHINE_DIR (so we can stick to unqualified Makefile.depend)

Include more stuff within the meta mode block, so that buildworld
can function normally.

Submitted by: John Van Horne

Cleanup some backwards compat stuff we do not need.
Introduce TARGET_OBJ_SPEC (derrived from TARGET_SPEC_VARS)
so we can build something like universe.

Add stage-install.sh so we can do away with the need to have
$STAGE_OBJTOP/include for src/include.

If building for "host" and we do not have Makefile.depend.host;
do not update it.
This allows us to avoid populating the tree with Makefile.depend.host
that aren't absolutely necessary.

We need .OBJDIR done earlier

Apply DEP_DIRDEPS_FILTER.${DEP_MACHINE} as needed.
Use DEP_DIRDEPS_FILTER.host to supress dependencies for host tools.

Control default .MAKE.DEPENDFILE indepentend of .MAKE.DEPENDFILE_PREFERENCE

Do no assume ${DESTDIR}/usr/include is always valid

If MAKESYSPATH contained .../ entry resolve it so that it still works
when we launch make from obj tree.
If we don't have sysroot support we need some c++ and clang specific
include dirs in the stage tree.

Use STAGE_OBJTOP for STAGE_SYMLINKS_DIR etc

Add TIME_STAMP

Need CXXFLAGS_LAST and LDFLAGS_LAST

Sync FreeBSD's bmake branch with Juniper's internal bmake branch.

Requested by: Simon Gerraty <sjg@juniper.net>