Merge commit 4bb2416d42eb from llvm-project (by Jessica Clarke):

[builtins][AArch64] Implement _sync out-of-line atomics

Whilst Clang does not use these, recent GCC does, and so on systems such
as FreeBSD that wish to use compiler-rt as the system runtime library
but also wish to support building programs with GCC these interfaces are
needed.

This is a light adaptation of the code committed to GCC by Sebastian Pop
<spop@amazon.com>, relicensed with permission for use in compiler-rt.

Fixes https://github.com/llvm/llvm-project/issues/63483

Reviewed By: sebpop, MaskRay

Differential Revision: https://reviews.llvm.org/D158536

Reviewed by: dim
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D41716

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Build compiler-rt against libunwind, not libcxxrt

Parts of compiler-rt are also built for libgcc_eh and libgcc_s, and
these were already pointing to the libunwind unwind.h. For the sake of
consistency, also build compiler-rt itself against the libunwind
unwind.h, not the libcxxrt one.

MFC after: 3 days

compilert-rt: build out-of-line LSE atomics helpers for aarch64

Both clang >= 12 and gcc >= 10.1 now default to -moutline-atomics for
aarch64. This requires a bunch of helper functions in libcompiler_rt.a,
to avoid link errors like "undefined symbol: __aarch64_ldadd8_acq_rel".

(Note: of course you can use -mno-outline-atomics as a workaround too,
but this would negate the potential performance benefit of the faster
LSE instructions.)

Bump __FreeBSD_version so ports maintainers can easily detect this.

PR: 257392
MFC after: 2 weeks

Rename NO_WERROR -> MK_WERROR=no

As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.

Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D27601

libcompiler_rt: stop building with stack smashing protection

libcompiler_rt implements certain functions that clang and gcc emit
calls to as part of their codegen (e.g. for extended width math). Build
it without stack smashing protection (SSP, -fstack-protector) in order
to support building binaries without SSP, especially the dynamic linker.
Besides, SSP is probably not very valuable in this library.

Reviewed by: arichardson, dim, kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D27786

Fix llvm-libunwind userspace build on ARM

GCC's libgcc exports a few ARM-specific symbols for ARM EABI, AEABI, or
EHABI or whatever it's called. Export the same ones from LLVM-libunwind's
libgcc_s, on ARM. As part of this, convert libgcc_s from a direct
Version.map to one constructed from component Symbol.map files. This allows
the ARM-specific Symbol.map to be included only on ARM.

Fix ARM-only oddities in struct name/aliases in LLVM-libunwind to match
non-ARM definitions and ARM-specific expectations in libcxxrt /
libcompiler_rt.

No functional change intended for non-ARM architectures.

This commit does not actually flip the switch for ARM defaults from libgcc
to llvm-libunwind, but makes it possible (to compile, anyway).

Gcc has incompatible internal declarations for __divtc3 and __multc3 as
defined in compiler-rt, but it has no option to silence its warning, so
make gcc warnings for libcompiler_rt non-fatal.

Noticed by: lwhsu
MFC after: 3 days

Remove an assembler flag, which is redundant since r309124. The upstream
took care of it by introducing a macro NO_EXEC_STACK_DIRECTIVE.

http://llvm.org/viewvc/llvm-project?rev=273500&view=rev

Reviewed by: dim

libcompiler_rt: move file list to Makefile.inc for reuse elsewhere

Also switch to the style used in the clang390-import branch to reduce
future conflicts.

Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8039

libcompiler_rt: use ${SRCTOP} for the top of the FreeBSD tree

Fix building some arm-specific primitives for libcompiler_rt. This was
an unfortunate search and replace error.

Make armv6 hard float abi by default. Kill armv6hf.
Allow CPUTYPE=soft to build the current soft-float abi libraries.
Add UPDATING entry to announce this.

Approved by: re@ (gjb)

First pass through library packaging.

Sponsored by: The FreeBSD Foundation

Annotate arm userspace assembler sources stating their tolerance to
the non-executable stack.

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation

compiler_rt: add floatunsitf for arm64

It provides unsigned integer to quad-precision conversion.

Sponsored by: The FreeBSD Foundation

compiler-rt: include 128-bit quad precision fp support only on arm64

Other architectures do not use quad precision long double and don't need
these runtime support routines.

Differential Revision: https://reviews.freebsd.org/D2252
Reviewed by: dim
Sponsored by: The FreeBSD Foundation

compiler-rt: add floatditf and floatunditf

These are long integer (di_int/du_int) to quad precision floating point
conversions. They may be reworked based on upstream discussion. These
versions are here to support arm64 world builds.

Reviewed by: ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2174

compiler-rt: Implement multc3 - quad-precision complex multiplication

This may be reworked based on upstream discussion. This version is here
to support arm64 world builds.

Reviewed by: ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2173

compiler-rt: Build additional quad precision floating point builtins

These are needed for arm64

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2160

Pull in r231965 from upstream compiler-rt trunk (by Jörg Sonnenberger):

Refactor float to integer conversion to share the same code.
80bit Intel/PPC long double is excluded due to lacking support
for the abstraction. Consistently provide saturation logic.
Extend to long double on 128bit IEEE extended platforms.

Initial patch with test cases from GuanHong Liu.
Reviewed by Steve Canon.

Differential Revision: http://reviews.llvm.org/D2804

Pull in r232107 from upstream compiler-rt trunk (by Ed Maste):

Use signed int implementation for __fixint

Requested by: emaste

With the update of compiler-rt we try to build a number of files that
don't build on some ARM platforms, provide symbols we already provide in
libc, or don't exist. Remove these from the build. Some of these may
return later on specific targets.

Differential Revision: https://reviews.freebsd.org/D1468
Reviewed by: dim, imp

Clean up detection of hard-float ABIs. As with big-endian in r272368 we
can check against arm*hf*.

Remove MK_ARM_EABI, the armeb issues have been fixed. The code to support
the oabi is still in the tree, but it is expected this will be removed
as developers work on surrounding code.

With this commit the ARM EABI is the only supported supported ABI by
FreeBSD on ARMa 32-bit processors.

X-MFC after: never
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D876

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Add a new ARM TARGET_ARCH, armv6hf. This is considered experimental.

This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.

armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
* WITH_ARM_EABI
* WITHOUT_GCC
* WITHOUT_GNUCXX

In addition, without an external toolchain, the following need to be left
as their default:
* WITH_CLANG
* WITH_CLANG_IS_CC

As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.

Unconditionally compile the __sync_* atomics support functions into compiler-rt
for ARM.
This is quite ugly, because it has to work around a clang bug that does not
allow built-in functions to be defined, even when they're ones that are
expected to be built as part of a library.

Reviewed by: ed

Build __clear_cache on ARM with clang now it supports it.

Let ARM use the custom tailored atomic intrinsics.

Use improved __sync_*() intrinsics for MIPS in userspace as well.

r251524 introduced custom tailored versions for MIPS of these functions
for kernel-space code. We can just reuse them in userspace as well.

Add __sync_synchronize().

This function can easily be implemented on top of the mb() macro
provided by <machine/atomic.h>.

Unbreak <stdatomic.h> on ARM + Clang.

Clang only supports atomic operations for ARMv6. For non-ARMv6, we still
need to emit these functions.

Clang's prototype for these functions slightly differs, as it is truly
based on GCC's documentation. It requires the use of signed types, but
also requires varargs. Still, we are not allowed to simply implement
this function directly. Cleverly work around this by implementing it
under a different name and using __strong_reference().

Enable libcompiler-rt on MIPS.

Originally we disabled libcompiler-rt on MIPS and SPARC64, because of an
issue where __clzdi2 and __ctzdi2 would cause endless recursion. This
bug has been fixed in r230021 already, but for some reason we only
switched to libcompiler-rt on SPARC64 -- not MIPS.

This means we can finally use <stdatomic.h> on all our architectures.

Add the __aeabi_mem* functions to compiler-rt as clang uses them.

Use the compiler-rt version __{u,}divsi3 and __{u,}modsi3 on ARM EABI

There should have been a tab after SRCS+=

Add the __aeabi_*divmod functions to the compiler-rt build

Get libcompiler-rt and libgcc building on ARM with clang.

* Don't provide clear_cache or the __sync_* functions on ARM with clang as
they are provided by clang as builtin functions.
* Tell clang it is aloud to compile some libgcc code using heinous GCC
extensions.

Switch sparc64 to using libcompiler_rt; since r230021 we have a workaround
in place allowing it to be used there and since r235388 (see also r235486)
we also have usable div/mod optimizations like libgcc has.

Add locally implemented atomic intrinsics to libcompiler_rt.

The built-in atomic operations are not implemented in our version of GCC
4.2 for the ARM and MIPS architectures. Instead of emitting locked
instructions, they generate calls to functions that can be implemented
in the C runtime.

Only implement the atomic operations that are used by <stdatomic.h> for
datatype sizes that are supported by atomic(9). This means that on these
architectures, we can only use atomic operations on 32-bits and 64-bits
variables, which is typically sufficient.

This makes <stdatomic.h> work on all architectures except MIPS, since
MIPS and SPARC64 still use libgcc. Converting these architectures to
libcompiler_rt is on my todo list.

Mark libcompiler_rt as not needed executable stack on powerpc.

Reviewed and tested by: nwhitehorn

Fix braino in r217101. -Wa is used to supply assembler flag to cc driver.

Noted by: Anonymous <swell.k gmail com>

On amd64 and i386, force assembler to mark objects compiled from the
assembler source for libcompiler_rt as not needed executable stack. This
is done with a hammer instead of properly marking each assembly file
with section .note.GNU-stack to avoid modifying contributed source.

Discussed with: ed

Switch mips architectures back to libgcc.

MIPS64 n64 binaries are broken with libcompiler_rt at this time.
Switch mips back to libgcc until the cause of breakage is analyzed
and fixed.

Revert to libgcc for sparc64.

I've had a report of a sparc64 system where cc1 generates illegal
instructions. We still have to diagnose this properly, but instead of
hosing all sparc64 boxes out there, fall back to libgcc to prevent more
damage.

Reported by: Florian Smeets

Set symbol visibility to hidden.

Not doing so may cause all sorts of random libraries to expose
libcompiler_rt's functions, which should of course not be done.

Discussed with: kan, kib