Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

clang: Build with -fno-strict-aliasing when using GCC

Somewhat ironically, there are strict aliasing violations in Clang,
which can result in the following assertion failure:

Assertion `*(NamedDecl **)&Data == ND && "PointerUnion mangles the NamedDecl pointer!"' failed.

Upstream's clang/CMakeLists.txt specifically (not LLVM as a whole)
passes -fno-strict-aliasing if the compiler is not Clang, and this fixes
the above issue.

This was seen when cross-building from Linux using a bootstrap
compiler, but likely also affects worlds built with a new enough
external GCC toolchain.

MFC after: 1 week
Reviewed by: dim
Differential Revision: https://reviews.freebsd.org/D31533

Move all sources from the llvm project into contrib/llvm-project.

This uses the new layout of the upstream repository, which was recently
migrated to GitHub, and converted into a "monorepo". That is, most of
the earlier separate sub-projects with their own branches and tags were
consolidated into one top-level directory, and are now branched and
tagged together.

Updating the vendor area to match this layout is next.

Completely revamp the way llvm, clang and lldb are built.

* Bootstrap llvm-tblgen and clang-tblgen with a minimal llvm static
library, that has no other dependencies.
* Roll up all separate llvm libraries into one big static libllvm.
* Similar for all separate clang and lldb static libraries.
* For all these libraries, generate their .inc files only once.
* Link all llvm tools (including extra) against the big libllvm.
* Link clang and clang-format against the big libllvm and libclang.
* Link lldb against the big libllvm, libclang and liblldb.

N.B.: This is work in progress, some details may still be missing.

It also heavily depends on bsd.*.mk's support for SRCS and DPSRCS with
relative pathnames, which apparently does not always work as expected.
For building llvm, clang and lldb though, it seems to work just fine.

The main idea behind this restructuring is maintainability and build
peformance. The previous large number of very small libraries, each
with their own generated files and dependencies was slow to traverse
and hard to understand.

Possible future improvements:
* Only build certain targets, e.g. for most regular users having just
one target will be fine. This will shave off some build time.
* Building the big llvm, clang and lldb libraries as shared (private)
libraries.
* Adding other components from the LLVM project, such as lld.

Update build glue for clang and the llvm/clang extras.

Reflect head is now 12.0-CURRENT.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Make armv6 hard float abi by default. Kill armv6hf.
Allow CPUTYPE=soft to build the current soft-float abi libraries.
Add UPDATING entry to announce this.

Approved by: re@ (gjb)

Limit Options.inc generation to desired targets

As mentioned in the Makefile there's an "atrocious" hack to generate a
different version of Options.inc.h, depending on the library being
built.

Remove the catch-all else case and limit it to specific libraries, so
that we don't accidentally use the Options.inc.h from clangdriver if a
future libary also uses Options.inc.h.

Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6209

Use the new bmake .dinclude feature to make these safe.

At least FAST_DEPEND won't even run 'make depend', so the code was
potentially broken with FAST_DEPEND anyhow. The .dinclude directive
will ignore missing files rather than make them be fatal.

Sponsored by: EMC / Isilon Storage Division

FAST_DEPEND: Always run depend via beforebuild which removes many hacks.

This will generate dependencies rather than depending on the previous behavior
of depending on the guessed OBJS: *.h dependecies or a user running
'make depend'.

Experimentation showed that depending only on headers was not enough and
prone to .ORDER errors. Downstream users may also have added
dependencies into beforedepend or afterdepend targets. The safest way to
ensure dependencies are generated before build is to run 'make depend'
beforehand rather than just depending on DPSRCS+SRCS.

Note that the OBJS_DEPEND_GUESS mechanism (a.k.a .if !exists(.depend) then
foo.o: *.h) is still useful as it improves incremental builds with missing
.depend.* files and allows 'make foo.o' to usually work, while this
'beforebuild: depend' ensures that the build will always find all dependencies.
The 'make foo.o' case has no means of a 'beforebuild' hook.

This also removes several hacks in the DIRDEPS_BUILD:
- NO_INSTALL_INCLUDES is no longer needed as it mostly was to work around
.ORDER problems with building the needed headers early.
- DIRDEPS_BUILD: It is no longer necesarry to track "local dependencies" in
Makefile.depend.

These were only in Makefile.depend for 'clean builds' since nothing would
generate the files due to skipping 'make depend' and early dependency
bugs that have been fixed, such as adding headers into SRCS for the
OBJS_DEPEND_GUESS mechanism. Normally if a .depend file does not exist then
a dependency is added by bsd.lib.mk/bsd.prog.mk from OBJS: *.h. However,
meta.autodep.mk creates a .depend file from created meta files and inserts
that into Makefile.depend. It also only tracks *.[ch] files though which can
miss some dependencies that are hooked into 'make depend'. This .depend
that is created then breaks incremental builds due to the !exists(.depend)
checks for OBJS_DEPEND_GUESS. The goal was to skip 'make depend' yet it only
really works the first time. After that files are not generated as expected,
which r288966 tried to address but was using buildfiles: rather than
beforebuild: and was reverted in r291725. As noted previously,
depending only on headers in beforebuild: would create .ORDER errors
in some cases.

meta.autodep.mk is still used to generate Makefile.depend though via:
gendirdeps: Makefile.depend
.END: gendirdeps

This commit allows removing all of the "local dependencies" in
Makefile.depend which cuts down on churn and removes some of the
arch-dependent Makefile.depend files.

The "local dependencies" were also problematic for bootstrapping.

Sponsored by: EMC / Isilon Storage Division

First part of updating llvm/clang build glue: getting llvm-tblgen,
clang-tblgen and clang itself built.

META MODE: Rework [bootstrapped] tools PATH support.

- Support more of the toolchain from TOOLSDIR.
- This also improves 'make bootstrap-tools' to pass, for example,
AS=/usr/bin/as to Makefile.inc1, which will tell cross-tools to use
external toolchain support and avoid building things we won't be using
in the build.
- Always set the PATH to contain the staged TOOLSDIR directories when
not building the bootstrap targets.

The previous version was only setting this at MAKE.LEVEL==0 and if the
TOOLSDIR existed. Both of these prevented using staged tools that were
built during the build though as DIRDEPS with .host dependencies, such
as the fix for needing usr.bin/localedef.host in r291311.
This is not a common tool so we must build and use it during the build,
and need to be prepared to change PATH as soon as it appears.

This should also fix the issue of host dependencies disappearing from
Makefile.depend and then reappearing due to the start of the fresh build not
having the directory yet, resulting in the tools that were built not actually
being used.
- Only use LEGACY_TOOLS while building in Makefile.inc1. After r291317
and r291546 there is no need to add LEGACY_TOOLS into the PATH for
the pseudo/targets/toolchain build.
- Because the pseudo/targets/toolchain will now build its own
[clang-]tblgen, the special logic in clang.build.mk is no longer needed.
- LEGACY_TOOLS is no longer used outside of targets/pseudo/bootstrap-tools
so is no longer passed into the environment in its build.

Sponsored by: EMC / Isilon Storage Division

Only DIRDEPS_BUILD should trigger this logic.

Missed in r290816.

Sponsored by: EMC / Isilon Storage Division

Remove redundant DPSRCS which were already in SRCS.

DPSRCS already contains all of SRCS.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Conditionalize the META_MODE tool handling on MK_META_MODE.

It was not being used outside of META_MODE but this should make it more clear
that it is only for META_MODE.

Sponsored by: EMC / Isilon Storage Division

For llvm/clang libraries, skip including tablegen-produced .d files when
the target is "make depend". This works around errors during
incremental make depend of some clang libraries, for example "don't know
how to make contrib/llvm/include/llvm/IR/IntrinsicsR600.td".

Reported by: emaste

Update llvm/clang build glue.

Followup to r278223, by only using -stdlib=libc++ when the compiler is
clang; not even recent versions of gcc support the -stdlib flag.

Noticed by: ngie

For now, add -stdlib=libc++ to the flags for building clang, since that
makes it easier to build head on stable/9, where libstdc++ is still the
default. We can revisit this when somebody will try to build base with
gcc 4.8.1 or higher, and its included libstdc++.

Reported by: rpaulo

Update llvm and clang library and binary Makefiles for 3.6.0 rc1.

Add the AArch64 llvm backend to the build to allow for early testing and
to ease any rework of how clang is built to take arm64 in to account.

Submitted by: andrew
Reviewed by: andrew, emaste
Differential Revision: https://reviews.freebsd.org/D1446

Disable the ObjC rewriter modules, since according to rdivacky these are
only used by Xcode. This shaves off some compilation time.

Update llvm and clang library Makefiles.

Fix the TARGET_ABI value clang uses. It shpuld be gnueabi on all ARM
soft-float architectures, and gnueabihf for hard-float.

Remove MK_ARM_EABI, the armeb issues have been fixed. The code to support
the oabi is still in the tree, but it is expected this will be removed
as developers work on surrounding code.

With this commit the ARM EABI is the only supported supported ABI by
FreeBSD on ARMa 32-bit processors.

X-MFC after: never
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D876

Replace DPADD with DPSRCS to fix "make checkdpadd"

Phabric: D625
Approved by: jmmv (mentor)
Reviewed by: dim
PR: 192734
MFC after: 2 weeks

In some scenarios, when generating llvm/clang .inc.h files, multiple
source files could be passed to tblgen or clang-tblgen, leading to a
"Too many positional arguments specified" error message. Fix this by
replacing the too-generic ${.ALLSRC} sources with explicit paths.

Reported by: rysto32@gmail.com, rodrigc
MFC after: 3 days

Ensure clang-tblgen in TOOLSDIR

Handle bootstrapping for host

Use the new -d option that was added to tblgen between llvm/clang 3.3
and 3.4 to generate dependency files for the '.inc.h' files generated
from .td files, and .sinclude those dependency files in clang.build.mk.

This will make future incremental builds of lib/clang and usr.bin/clang
work correctly, whenever any of the .td files get modified.

Note that this will not fix any problems with incremental builds from
*before* this revision, since there will not yet be any generated
dependency files. A quick workaround is to run the following:

find /usr/obj -type f -name '*.inc.h' | xargs rm

and then a regular incremental buildworld (e.g. with -DNO_CLEAN).

MFC after: 3 days

Eliminate EARLY_BUILD flag. It is redundant and means MK_CLANG_FULL=no
and MK_LLDB=no, so set those explicitly (now that we can do
that). Simplify tests for these variables as well, since we know they
will always be defined regardless of the phase of the build.

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Add a new ARM TARGET_ARCH, armv6hf. This is considered experimental.

This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.

armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
* WITH_ARM_EABI
* WITHOUT_GCC
* WITHOUT_GNUCXX

In addition, without an external toolchain, the following need to be left
as their default:
* WITH_CLANG
* WITH_CLANG_IS_CC

As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.

Add Makefile glue to build the Sparc backend libraries and link them
into the clang executable.

Use the tblgen etc we built for host.

Make sure we trigger generation of headers

Bump OS versions in the toolchain triples to 11.0, and bump the
__FreeBSD_cc_version predefined macros in clang and gcc.

Approved by: re (gjb)

Allow us to build clang for ARM EABI. Clang and llvm use the
arm-gnueabi-freebsd10.0 triple for EABI. Use this when we are on arm or
armv6 and are building for EABI.

Reviewed by: dim

Pull in r170135 from upstream clang trunk:

Dont use/link ARCMT, StaticAnalyzer and Rewriter to clang when the user
specifies not to. Dont build ASTMatchers with Rewriter disabled and
StaticAnalyzer when it's disabled.

Without all those three, the clang binary shrinks (x86_64) from ~36MB
to ~32MB (unstripped).

To disable these clang components, and get a smaller clang binary built
and installed, set WITHOUT_CLANG_FULL in src.conf(5). During the
initial stages of buildworld, those extra components are already
disabled automatically, to save some build time.

MFC after: 1 week

Sprinkle a bit of style.Makefile(5) across various clang Makefiles. No
functional changes.

MFC after: 3 days

Revert r234356 for now, as it leads to run-time problems on 32-bit
PowerPC. Note this will break world.

Reported by: andreast
Pointy hat to: dim

Work around an issue on 32-bit PowerPC, where clang executable can get
too big, causing 'relocation truncated to fit' errors at link time.

Reviewed by: nwhitehorn

Instead of forcing -O1 on PowerPC for LLVM and clang, fix the actual
problem by adding -fno-strict-aliasing to CFLAGS. Since this is a global
issue that just happened to manifest on PowerPC, add this to CFLAGS
unconditionally.

MFC after: 1 week

Connect LLVM/clang to the build on powerpc64. After the binutils 2.17.50
import, it works without issue.

Update llvm/clang's target triple (confusingly named LLVM_HOSTTRIPLE) to
$arch-unknown-freebsd10.0.

On amd64, change clang's default triple to 'x86_64-unknown-freebsd9.0',
similar to what we do for binutils. When clang's default triple starts
with 'amd64-', it does not pass a proper -target-cpu option to its
first stage.

This can lead to problems, for example when structs are memcpy'd, and
clang erroneously assumes they are 16-byte aligned. It will then use
the 'movaps' SSE instruction to implement the copy, which results in a
bus error if the struct is really 8-byte aligned.

I encountered this issue when gcc's /usr/libexec/cc1 started crashing
with SIGBUS, after rebuilding world with clang ToT, but it also affects
the version of clang that we have in the tree. We were just lucky until
now, apparently. :)

Don't use -fno-exceptions or -fno-rtti for .c files, when building
llvm/clang.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Modify clang so that when TOOLS_PREFIX is defined we register the
CLANG_PREFIX macro. This changes the default header search path when we
are building clang as part of cross-tools.

Submitted by: Dimitry Andric <dimitry at andric.com>
Reviewed by: freebsd-current

Build clang libraries by default with -fno-exceptions and with
-fno-rtti. The clang libaries that really use exceptions and virtual
functions can enable LLVM_REQUIRES_EH and LLVM_REQUIRES_RTTI
respectively. This saves space on the resulting binaries and follows
what's being done upstream.

Submitted by: Dimitry Andric <dimitry at andric.com>

Remove unneeded files from the build.

I used the following command to determine which source files were
unneeded:

| for i in `find lib/clang -name '*.o'`
| do
| MATCHES="`(nm -g --defined-only $i; nm -g --defined-only \
| usr.bin/clang/clang/clang) | sed -e 's/.* //' | \
| sort | uniq -d | wc -l`"
| [ $MATCHES -eq 0 ] && echo "$i: unneeded"
| done

This should slightly improve the build times.

Unbreak Clang on PowerPC.

It seems GCC 4.2.1 on PowerPC miscompiles Clang, causing it to crash
when building even simple Hello World applications. Switch back to -O1
for this architecture.

Submitted by: nwhitehorn

Disable usage of posix_spawn() inside LLVM.

Even though it's nice to use posix_spawn() instead of manually using
fork()/exec(), it's better to disable this. FreeBSD 7 doesn't support
this interface. When enabled, we can't build tblgen, which prevents us
from building FreeBSD 9 on 7.

Tested by: raj

Do the branding right this time.

I've looked at other places in the source tree where CLANG_VENDOR is
used and I suspect it might not be safe to use newlines here.
CLANG_VENDOR should just be defined to "FreeBSD ", just like the latest
Clang preview in OS X uses "Apple ". Properly use SVN_REVISION to define
it to the imported revision of Clang. I do want to have a date in there,
so slightly modify the code to support CLANG_VENDOR_SUFFIX.

Brand our Clang binary.

That way it's easier for the Clang folks to keep track of what we're
doing.

Import the build makefiles for clang/LLVM.

Approved by: ed (mentor)