Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Build libclang also if LLDB is enabled

LLDB depends on libclang as it uses Clang as the expression parser.
Previously setting WITHOUT_CLANG but leaving LLDB enabled (as default)
resulted in a build failure.

Users who set WITHOUT_CLANG in order to reduce build time or size
might want to set WITHOUT_LLDB in addition to WITHOUT_CLANG, or use
WITHOUT_TOOLCHAIN instead.

PR: 260993
Reported by: eugen
Reviewed by: dim
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Add WITH_LLVM_BINUTILS to install LLVM binutils instead of Elftoolchain

When WITH_LLVM_BINUTILS is set, we will install the LLVM binutils as
ar/ranlib/nm/objcopy/etc. instead of the elftoolchain ones.
Having the LLVM binutils instead of the elftoolchain ones allows us to use
features such as LTO that depend on binutils that understand LLVM IR.
Another benefit will be an improved user-experience when compiling with
AddressSanitizer, since ASAN does not symbolize backtraces correctly if
addr2line is elftoolchain addr2line instead of llvm-symbolizer.
See https://lists.freebsd.org/archives/freebsd-toolchain/2021-July/000062.html
for more details.

This is currently off by default but will be turned on by default at some
point in the near future.

Reviewed By: emaste

Differential Revision: https://reviews.freebsd.org/D31060

Completely revamp the way llvm, clang and lldb are built.

* Bootstrap llvm-tblgen and clang-tblgen with a minimal llvm static
library, that has no other dependencies.
* Roll up all separate llvm libraries into one big static libllvm.
* Similar for all separate clang and lldb static libraries.
* For all these libraries, generate their .inc files only once.
* Link all llvm tools (including extra) against the big libllvm.
* Link clang and clang-format against the big libllvm and libclang.
* Link lldb against the big libllvm, libclang and liblldb.

N.B.: This is work in progress, some details may still be missing.

It also heavily depends on bsd.*.mk's support for SRCS and DPSRCS with
relative pathnames, which apparently does not always work as expected.
For building llvm, clang and lldb though, it seems to work just fine.

The main idea behind this restructuring is maintainability and build
peformance. The previous large number of very small libraries, each
with their own generated files and dependencies was slow to traverse
and hard to understand.

Possible future improvements:
* Only build certain targets, e.g. for most regular users having just
one target will be fine. This will shave off some build time.
* Building the big llvm, clang and lldb libraries as shared (private)
libraries.
* Adding other components from the LLVM project, such as lld.

Update build glue for lldb. Also comment out parts of the
initialization and termination code which reference plugins and
components that we don't use.

Update build glue for clang and the llvm/clang extras.

Add clang-format under WITH_CLANG_EXTRAS.

Reviewed by: dim
Approved by: re (gjb)
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D6856

Update build infrastructure for LLDB 3.8

Next part of updating llvm/clang build glue: getting the
WITH_CLANG_EXTRAS tools built.

First part of updating llvm/clang build glue: getting llvm-tblgen,
clang-tblgen and clang itself built.

libllvmmirparser and libllvmlibdriver are only used in usr.bin/clang/llc and
usr.bin/clang/llvm-ar, respectively, when MK_CLANG_EXTRAS is yes.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Minimize the number of files compiled for clang only (e.g. when neither
WITH_CLANG_EXTRAS nor WITH_LLDB is in effect).

Update llvm/clang build glue.

Update Makefiles and other build glue for llvm/clang 3.7.0, as of trunk
r238337.

Update FreeBSD LLDB build for 3.6 update

Sponsored by: DARPA, AFRL

Update llvm and clang library and binary Makefiles for 3.6.0 rc1.

Add the AArch64 llvm backend to the build to allow for early testing and
to ease any rework of how clang is built to take arm64 in to account.

Submitted by: andrew
Reviewed by: andrew, emaste
Differential Revision: https://reviews.freebsd.org/D1446

Update build for LLDB snapshot at upstream rev 216948

Sponsored by: DARPA, AFRL

Update llvm and clang library Makefiles.

Eliminate EARLY_BUILD flag. It is redundant and means MK_CLANG_FULL=no
and MK_LLDB=no, so set those explicitly (now that we can do
that). Simplify tests for these variables as well, since we know they
will always be defined regardless of the phase of the build.

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
all the SUBDIR entries in parallel, instead of serially. Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.

This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.

I tested this on a 24-core machine, with make -j48 buildworld (N = 6):

before stddev after stddev
======= ====== ======= ======
real time 1741.1 16.5 959.8 2.7
user time 12468.7 16.4 14393.0 16.8
sys time 1825.0 54.8 2110.6 22.8

(user+sys)/real 8.2 17.1

E.g. the build was approximately 45% faster in real time. On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive. But at least you can now almost max out a machine with
buildworld!

Submitted by: jilles
MFC after: 2 weeks

Add Makefile glue to build the Sparc backend libraries and link them
into the clang executable.

Enable llvm's integrated assembler for PowerPC, since it should now be
good enough for typical usage.

Requested by: rdivacky
MFC after: 1 week

Add LLDB bmake infrastructure

This connects LLDB to the build, but it is disabled by default. Add
WITH_LLDB= to src.conf to build it.

Note that LLDB requires a C++11 compiler so is disabled on platforms
using GCC.

Approved by: re (gjb)
Sponsored by: DARPA, AFRL

Pull in r170135 from upstream clang trunk:

Dont use/link ARCMT, StaticAnalyzer and Rewriter to clang when the user
specifies not to. Dont build ASTMatchers with Rewriter disabled and
StaticAnalyzer when it's disabled.

Without all those three, the clang binary shrinks (x86_64) from ~36MB
to ~32MB (unstripped).

To disable these clang components, and get a smaller clang binary built
and installed, set WITHOUT_CLANG_FULL in src.conf(5). During the
initial stages of buildworld, those extra components are already
disabled automatically, to save some build time.

MFC after: 1 week

Cleanup a few duplicate llvm libs that snuck in with the recent update.

Noticed by: rdivacky

When the make target is 'install', don't descend into the clang
libraries subdirectories since there's nothing to do there. This saves
us quite a few seconds off installworld, esp. if the disk I/O is slow.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Import the build makefiles for clang/LLVM.

Approved by: ed (mentor)