Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r234840:
Split the symlook_obj1 into a loop iterating over the ELF object symbol
hash elements, and a helper matched_symbol() which match the given hash
entry and request, performing needed type and version checks.

MFC r234841:
Add GNU hash support for rtld.

MFC r235054:
Work around a situation where symlook_obj() could be called for the
object for which digest_dynamic1() was not done yet. Just return
EINVAL and do not try to dereference NULL buckets hash array.

MFC r233231:
Fix several problems with our ELF filters implementation.

Do not relocate twice an object which happens to be needed by loaded
binary (or dso) and some filtee opened due to symbol resolution when
relocating need objects. Record the state of the relocation
processing in Obj_Entry and short-circuit relocate_objects() if
current object already processed.

Do not call constructors for filtees loaded during the early
relocation processing before image is initialized enough to run
user-provided code. Filtees are loaded using dlopen_object(), which
normally performs relocation and initialization. If filtee is
lazy-loaded during the relocation of dso needed by the main object,
dlopen_object() runs too earlier, when most runtime services are not
yet ready.

Postpone the constructors call to the time when main binary and
depended libraries constructors are run, passing the new flag
RTLD_LO_EARLY to dlopen_object(). Symbol lookups callers inform
symlook_* functions about early stage of initialization with
SYMLOOK_EARLY. Pass flags through all functions participating in
object relocation.

Use the opportunity and fix flags argument to find_symdef() in
arch-specific reloc.c to use proper name SYMLOOK_IN_PLT instead of
true, which happen to have the same numeric value.

MFC r233777 (by kan):
Do not try to adjust stacks if dlopen_object is called too early.

MFC r233778 (by kan):
Remove extra blank line from revious commit.

MFC note: the ARM and MIPS TLS support is not merged back, so the chunks
from r233231 which fix misuse of flags in calls to find_symdef() in
the corresponding relocation type handlers were not applied. When TLS
support is merged, the rest of r233231 should be applied too.

MFC r232831:
Add support for preinit, init and fini arrays to rtld.
Only binaries marked with proper ABI note gets array ctr/dtrs called.

MFC r232856:
When iterating over the dso program headers, the object is not initialized
yet, and object segments are not yet mapped. Only parse the notes that
appear in the first page of the dso (as it should be anyway), and use
the preloaded page content.

MFC r232857 (by dim):
Fix a warning/error with clang.

MFC r232859 (by dim):
Amend r232857, now dropping the casts entirely, as they were not
necessary at all.

MFC r228435:
Add support for STT_GNU_IFUNC and R_MACHINE_IRELATIVE GNU extensions to
rtld on 386 and amd64.

MFC r228503:
Postpone the resolution for irelative/ifunc right before initializers
are called, and drop bind lock around calls to dispatcher. Use
initlist to iterate over the objects instead of the ->next, due to
drop of the bind lock in iteration.

For i386/reloc.c:reloc_iresolve(), fix calculation of the dispatch
function address for dso, by taking into account possible non-zero
relocbase.

MFC r228635 (by nwhitehorn):
Fix RTLD on PowerPC after r228435. Changing the order of init_pltgot()
caused the icache to be invalidated at the wrong time, resulting in
an icache full of nonsense in the PLT section.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

- Add support for TLS relocations.
- Emitt an error when encountering an unsupported and in case of the
kernel also for unaligned relocations.
- Fix R_SPARC_LOX10 relocations. Apparently these are hardly ever used.

- Remove clause 3 and 4 from TNF licenses. [1]
- Add the _RF_X committed in r212998 also to the tables in the sparc64
reloc.c in order reduce differences between the kernel and the userland
source. This results in no functional change though.
- Fix further inconsistencies in the abbreviations of the names of the
relocations.
- Further whitespace fixes.

Obtained from: NetBSD [1]

- With the addition of TLS support binutils started to make the addend
values for resolved symbols relative to relocbase instead of sections
so detect this case and handle as appropriate, which allows using
kernel modules linked with affected versions of binutils. Actually I
think this is a bug in binutils but given that apparently nobody
complained for nearly six years and powerpc has basically the same
workaround I decided to put it in for the sparc64 kernel, too.
- Fix R_SPARC_HIX22 relocations. Apparently these are hardly ever used.

- Consistently abbreviate the names of the relocations.
- End sentences with dots.
- Fix whitespace.

When loading dso without PT_GNU_STACK phdr, only call
__pthread_map_stacks_exec() on architectures that allow executable
stacks.

Reported and tested by: marcel (ia64)

Implement support for ELF filters in rtld. Both normal and auxillary
filters are implemented.

Filtees are loaded on demand, unless LD_LOADFLTR environment variable
is set or -z loadfltr was specified during the linking. This forces
rtld to upgrade read-locked rtld_bind_lock to write lock when it
encounters an object with filter during symbol lookup.

Consolidate common arguments of the symbol lookup functions in the
SymLook structure. Track the state of the rtld locks in the
RtldLockState structure. Pass local RtldLockState through the rtld
symbol lookup calls to allow lock upgrades.

Reviewed by: kan
Tested by: Mykola Dzham <i levsha me>, nwhitehorn (powerpc)

Only use the cache after the early stage of loading. This is
because calling mmap() etc. may use GOT which is not set up
yet. Use calloc() instead of mmap() in cases where this
was the case before (sparc64, powerpc, arm).

Submitted by: Dimitry Andric (dimitry andric com)
Reviewed by: kan
Approved by: ed (mentor)

Now that the kernel defines CACHE_LINE_SIZE in machine/param.h, use
that definition in the custom locking code for the run-time linker
rather than local definitions.

Pointed out by: tinderbox
MFC after: 2 weeks

For un-prototyped static inline functions declared in pthread_md.h on
sparc64, use ANSI function headers and specifically indicate the lack of
arguments with 'void'. Otherwise, warnings are generated at WARNS=3 for
libkse, leading to a compile failure with -Werror.

- Fix the handling of R_SPARC_OLO10, which is a bit of a special case
in the way we implement handling of relocations.
As for the kernel part this fixes the loading of lots of modules,
which failed to load due to unresolvable symbols when built after
the GCC 4.2.0 import. This wasn't due to a change in GCC itself
though but one of several changes in configuration done along the
import. Specfically, HAVE_AS_REGISTER_PSEUDO_OP, which causes GCC
to denote global registers used for scratch purposes and in turn
GAS uses R_SPARC_OLO10 relocations for, is now defined.
While at it replace some more ELF_R_TYPE which should have been
ELF64_R_TYPE_ID but didn't cause problems so far.
- Sync a sanity check between kernel and rtld(1) and change it to be
maintenance free regarding the type used for the lookup table.
- Sprinkle const on lookup tables.
- Use __FBSDID.

Reported and tested by: yongari
MFC after: 5 days

Cache does not serve any purpose when rtld is relocating itself, do
not bother allocating one.

Submitted by: kan
Approved by: re (bmah)

Fix TLS on sparc64 for statically and dynamically linked binaries
Approved by: rwatson (mentor)
Reviewed by: jmg and marcel

Allocate space for thread pointer, this allows thread library to access
its pointer from begin, and simplifies _get_curthread() in libthr.

Implement ELF symbol versioning using GNU semantics. This code aims
to be compatible with symbol versioning support as implemented by
GNU libc and documented by http://people.redhat.com/~drepper/symbol-versioning
and LSB 3.0.

Implement dlvsym() function to allow lookups for a specific version of
a given symbol.

Make our ELF64 type definitions match standards. In particular this
means:
o Remove Elf64_Quarter,
o Redefine Elf64_Half to be 16-bit,
o Redefine Elf64_Word to be 32-bit,
o Add Elf64_Xword and Elf64_Sxword for 64-bit entities,
o Use Elf_Size in MI code to abstract the difference between
Elf32_Word and Elf64_Word.
o Add Elf_Ssize as the signed counterpart of Elf_Size.

MFC after: 2 weeks

MFNetBSD: src/libexec/ld.elf_so/arch/sparc64/mdreloc.c 1.26 (partial)

Fix an obvious bug in the 64-bit PLT fixup: the SLLX was by 12 bits, when
it should be 32.

MFC after: 1 month

Remove these unused files before any other archs include the same bogus
file.

Add support for Thread Local Storage.

Fix the problem that surfaced with the new binutils import on sparc64
(and that is for now being worked around by a binutils patch).

The rtld code tested &_DYNAMIC against 0 to see whether rtld itself
was built as PIC or not. While the sparc64 MD code did not rely
on the preset value of the GOT slot for _DYNAMIC any more due
to previous binutils changes, it still used to not be 0, so
that this check did work. The new binutils do however initialize
this slot with 0. As a consequence, rtld would not properly initialize
itself and crash.
Fix that by introducing a new macro, RTLD_IS_DYNAMIC, to take the role
of this test. For sparc64, it is implemented using the rtld_dynamic()
code that was already there. If an architecture does not provide its
own implementation, we default to the old check.

While being there, mark _DYNAMIC as a weak symbol in the sparc64
rtld_start.S. This is needed in the LDSCRIPT case, which is however
not currently supported for want of an actual ldscript.

Sanity checked with md5 on alpha, amd64, i386 and ia64.

Avoid using the global offset table to get the address of _DYNAMIC in
rtld. When _DYNAMIC is referenced normally from C the global offset
table is used implicitly, but newer versions of binutils don't initialize
it statically in the binary, so this doesn't work until rtld is relocated,
which _DYNAMIC is needed for... So, as on other systems with the same
problem, we disassemble a call instruction to _DYNAMIC in order to get
its address.

Allow threading libraries to register their own locking
implementation in case default one provided by rtld is
not suitable.

Consolidate various identical MD lock implementation into
a single file using appropriate machine/atomic.h.

Approved by: re (scottl)

No need to zero fill memory, mmapped anonymously. Kernel will
return pre-zeroed pages itself.

Noticed by: jake

Fix rtld to handle SPARC_R_UA{16,64} relocations correctly.

Approved by: re (rwatson)

Fix the handling of high PLT entries (> 32764) on sparc64. This requires
additional arguments to reloc_jmpslot(), which is why MI code and MD code
of other platforms had to be changed.

Reviewed by: jake
Approved by: re

<machine/atomic.h> requires <sys/types.h>.

Reviewed by: jake, mike

Return an error if a symbol is not found in reloc_jmpslots() instead of
crashing.

Add END markers to asm functions so that debuggers can find their size.

Remove the nanosleep calls from the spin loops in the locking code.
They provided little benefit (if any) and they caused some problems
in OpenOffice, at least in post-KSE -current and perhaps in other
environments too. The nanosleep calls prevented the profiling timer
from advancing during the spinloops, thereby preventing the thread
scheduler from ever pre-empting the spinning thread. Alexander
Kabaev diagnosed this problem, Martin Blapp helped with testing,
and Matt Dillon provided some helpful suggestions.

This is a short-term fix for a larger problem. The use of spinlocking
isn't guaranteed to work in all cases. For example, if the spinning
thread has higher priority than all other threads, it may never be
pre-empted, and the thread holding the lock may never progress far
enough to release the lock. On the other hand, spinlocking is the
only locking that can work with an arbitrary unknown threads package.

I have some ideas for a much better fix in the longer term. It
would eliminate all locking inside the dynamic linker by making it
safe for symbol lookups and lazy binding to proceed in parallel
with a call to dlopen or dlclose. This means that the only mutual
exclusion needed would be to prevent multiple simultaneous calls
to dlopen and/or dlclose. That mutual exclusion could be put into
the native pthreads library. Applications using foreign threads
packages would have to make their own arrangements to ensure that
they did not have multiple threads in dlopen and/or dlclose -- a
reasonable requirement in my opinion.

MFC after: 3 days

Add needed include of mman.h to fix sparc64 buildworld.

The last bits of the alloca -> mmap fix. IA64 and SPARC64 (current only).
Untested (testing request went unanswered), but sparc64 is not expected to
cause problems. IA64 is not expected to cause problems but the patch was
slightly more complex so the possibility exists.

Approved by: jdp

rtld support for sparc64.

Largely obtained from: netbsd
Submitted by: jake, tmm