- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Implement support for the interpose dso flag.

Requested by: bf
Reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (glebius)

Import the DragonFly BSD commit 4f0bc915b65fcf5a23214f6d221d65c80be68ad4
by John Marino <draco@marino.st>, with the following (edited) commit
message
Date: Sat, 24 Mar 2012 06:40:50 +0100
Subject: [PATCH 1/1] rtld: Implement DT_RUNPATH and -z nodefaultlib

DT_RUNPATH is incorrectly being considered as an alias of DT_RPATH. The
purpose of DT_RUNPATH is to have two different types of rpath: one that
can be overridden by the environment variable LD_LIBRARY_PATH and one that
can't. With the currently implementation, LD_LIBRARY_PATH will always
trump any embedded rpath or runpath tags.

Current path search order by rtld:
==================================
LD_LIBRARY_PATH
DT_RPATH / DT_RUNPATH (always the same)
ldconfig hints file (default: /var/run/ld-elf.so.hints)
/usr/lib

New path search order by rtld:
==============================
DT_RPATH of the calling object if no DT_RUNPATH
DT_RPATH of the main binary if no DT_RUNPATH and binary isn't calling obj
LD_LIBRARY_PATH
DT_RUNPATH
ldconfig hints file
/usr/lib

The new path search matches how the linux runtime loader works. The other
major added feature is support for linker flag "-z nodefaultlib". When
this flag is passed to the linker, rtld will skip all references to the
standard library search path ("/usr/lib" in this case but it could handle
more color delimited paths) except in DT_RPATH and DT_RUNPATH.

New path search order by rtld with -z nodefaultlib flag set:
============================================================
DT_RPATH of the calling object if no DT_RUNPATH
DT_RPATH of the main binary if no DT_RUNPATH and binary isn't calling obj
LD_LIBRARY_PATH
DT_RUNPATH
ldconfig hints file (skips all references to /usr/lib)

FreeBSD notes:
- we fixed some bugs which were submitted to DragonFly and merged there
as commit 1ff8a2bd3eb6e5587174c6a983303ea3a79e0002;
- we added LD_LIBRARY_PATH_RPATH environment variable to switch to
the previous behaviour of considering DT_RPATH a synonym for DT_RUNPATH;
- the FreeBSD default search path is /lib:/usr/lib and not /usr/lib.

Reviewed by: kan
MFC after: 1 month
MFC note: flip the ld_library_path_rpath default value for stable/9

Add GNU hash support for rtld.

Based on dragonflybsd support for GNU hash by John Marino <draco marino st>
Reviewed by: kan
Tested by: bapt
MFC after: 2 weeks

Split the symlook_obj1 into a loop iterating over the ELF object symbol
hash elements, and a helper matched_symbol() which match the given hash
entry and request, performing needed type and version checks.

Based on dragonflybsd support for GNU hash by John Marino <draco marino st>
Reviewed by: kan
Tested by: bapt
MFC after: 2 weeks

Prevent rtld_verify_object_versions() from being called several times
for the same object. This can happen when object is a dependency of the
dlopen()ed dso. When called several times, we waste time due to unneeded
processing, and memory, because obj->vertab is allocated anew on each
iteration.

Reviewed by: kan
MFC after: 2 weeks

Remove superfluous extern keywords.

MFC after: 2 weeks

Use xmalloc() instead of malloc() in the places where malloc() calls
are assumed to not fail.

Make the xcalloc() calling conventions follow the calloc(3) calling
conventions and replace unchecked calls to calloc() with calls to
xcalloc().

Remove redundand declarations from xmalloc.c, which are already
present in rtld.h.

Reviewed by: kan
Discussed with: bde
MFC after: 2 weeks

Fix several problems with our ELF filters implementation.

Do not relocate twice an object which happens to be needed by loaded
binary (or dso) and some filtee opened due to symbol resolution when
relocating need objects. Record the state of the relocation
processing in Obj_Entry and short-circuit relocate_objects() if
current object already processed.

Do not call constructors for filtees loaded during the early
relocation processing before image is initialized enough to run
user-provided code. Filtees are loaded using dlopen_object(), which
normally performs relocation and initialization. If filtee is
lazy-loaded during the relocation of dso needed by the main object,
dlopen_object() runs too earlier, when most runtime services are not
yet ready.

Postpone the constructors call to the time when main binary and
depended libraries constructors are run, passing the new flag
RTLD_LO_EARLY to dlopen_object(). Symbol lookups callers inform
symlook_* functions about early stage of initialization with
SYMLOOK_EARLY. Pass flags through all functions participating in
object relocation.

Use the opportunity and fix flags argument to find_symdef() in
arch-specific reloc.c to use proper name SYMLOOK_IN_PLT instead of
true, which happen to have the same numeric value.

Reported and tested by: theraven
Reviewed by: kan
MFC after: 2 weeks

Rtld on diet 3.

Stop using strerror(3) in rtld, which brings in msgcat and stdio.
Directly access sys_errlist array of errno messages with private
rtld_strerror() function.

Now,
$ size /libexec/ld-elf.so.1
text data bss dec hex filename
96983 2480 8744 108207 1a6af /libexec/ld-elf.so.1

Reviewed by: dim, kan
MFC after: 2 weeks

Add support for preinit, init and fini arrays. Some ABIs, in
particular on ARM, do require working init arrays.

Traditional FreeBSD crt1 calls _init and _fini of the binary, instead
of allowing runtime linker to arrange the calls. This was probably
done to have the same crt code serve both statically and dynamically
linked binaries. Since ABI mandates that first is called preinit
array functions, then init, and then init array functions, the init
have to be called from rtld now.

To provide binary compatibility to old FreeBSD crt1, which calls _init
itself, rtld only calls intializers and finalizers for main binary if
binary has a note indicating that new crt was used for linking. Add
parsing of ELF notes to rtld, and cache p_osrel value since we parsed
it anyway.

The patch is inspired by init_array support for DragonflyBSD, written
by John Marino.

Reviewed by: kan
Tested by: andrew (arm, previous version), flo (sparc64, previous version)
MFC after: 3 weeks

Add support for GNU RELRO.

Submitted by: John Marino <draco marino st>
MFC after: 2 weeks

Spelling fixes for libexec/

Add support for STT_GNU_IFUNC and R_MACHINE_IRELATIVE GNU extensions to
rtld on 386 and amd64. This adds runtime bits neccessary for the use
of the dispatch functions from the dynamically-linked executables and
shared libraries.

To allow use of external references from the dispatch function, resolution
of the R_MACHINE_IRESOLVE relocations in PLT is postponed until GOT entries
for PLT are prepared, and normal resolution of the GOT entries is finished.
Similar to how it is done by GNU, IRELATIVE relocations are resolved in
advance, instead of normal lazy handling for PLT.

Move the init_pltgot() call before the relocations for the object are
processed.

MFC after: 3 weeks

Typo.

MFC after: 3 days

Rtld links with the specially built pic static libc library to get some
C runtime services, like printf(). Unfortunately, the multithread-safeness
measures in the libc do not work in rtld environment.

Rip the kernel printf() implementation and use it in the rtld instead of
libc version. This printf does not require any shared global data and thus
is mt-safe. Systematically use rtld_printf() and related functions, remove
the calls to err(3).

Note that stdio is still pulled from libc due to libmap implementaion using
fopen(). This is safe but unoptimal, and can be changed later.

Reported and tested by: pgj
Diagnosed and reviewed by: kan (previous version)
Approved by: re (bz)

Use sigsetjmp/siglongjmp with disabled signal mask access for
lock upgrade in rtld. There is no need to care about the mask,
which causes a lot of unneeded sigprocmask(2) calls during each
symbol lookup.

In rtld, read the initial stack access mode from AT_STACKPROT as set
by kernel, and parse PT_GNU_STACK phdr from linked and loaded dsos.

If the loaded dso requires executable stack, as specified by PF_X bit
of p_flags of PT_GNU_STACK phdr, but current stack protection does not
permit execution, the __pthread_map_stacks_exec symbol is looked up
and called. It should be implemented in libc or threading library and
change the protection mode of all thread stacks to be executable.

Provide a private interface _rtld_get_stack_prot() to export the stack
access mode as calculated by rtld.

Reviewed by: kan

Implement support for ELF filters in rtld. Both normal and auxillary
filters are implemented.

Filtees are loaded on demand, unless LD_LOADFLTR environment variable
is set or -z loadfltr was specified during the linking. This forces
rtld to upgrade read-locked rtld_bind_lock to write lock when it
encounters an object with filter during symbol lookup.

Consolidate common arguments of the symbol lookup functions in the
SymLook structure. Track the state of the rtld locks in the
RtldLockState structure. Pass local RtldLockState through the rtld
symbol lookup calls to allow lock upgrades.

Reviewed by: kan
Tested by: Mykola Dzham <i levsha me>, nwhitehorn (powerpc)

If dlopen() is called for the dso that has been already loaded as a
dependency, then the dso never has its DAG initialized. Empty DAG
makes ref_dag() call in dlopen() a nop, and the dso refcount is off
by one.

Initialize the DAG on the first dlopen() call, using a boolean flag
to prevent double initialization.

From the PR (edited):
Assume we have a library liba.so, containing a function a(), and a
library libb.so, containing function b(). liba.so needs functionality
from libb.so, so liba.so links in libb.so.

An application doesn't know about the relation between these libraries,
but needs to call a() and b(). It dlopen()s liba.so and obtains a
pointer to a(), then it dlopen()s libb.so and obtains a pointer to b().

As soon as the application doesn't need a() anymore, it dlclose()s liba.so.

Expected result: the pointer to b() is still valid and can be called
Actual result: the pointer to b() has become invalid, even though the
application did not dlclose() the handle to libb.so. On calling b(), the
application crashes with a segmentation fault.

PR: misc/151861
Based on patch by: jh
Reviewed by: kan
Tested by: Arjan van Leeuwen <freebsd-maintainer opera com>
MFC after: 1 week

Check for undefined weak symbols during PLT binding on powerpc64, and do
not attempt to copy NULL function descriptors. This fixes LD_BIND_NOW on
powerpc64 after r211706.

Reviewed by: kib

Allow to load not-openable dso when tracing. This fixes ldd on such dso or
dso linked to non-openable object.
Remove '\n' at the end of error message.
End comments with dot.

MFC after: 3 weeks (together with r199829)

Implement rtld part of the support for -z nodlopen (see ld(1)).

Reviewed by: kan
MFC after: 3 weeks

Allow order of initialization of loaded shared objects to be
altered through their .init code. This might happen if init
vector calls dlopen on its own and that dlopen causes some not
yet initialized object to be initialized earlier as part of that
dlopened DAG.

Do not reset module reference counts to zero on final fini vector
run when process is exiting. Just add an additional parameter to
force fini vector invocation regardless of current reference count
value if object was not destructed yet. This allows dlclose called
from fini vector to proceed normally instead of failing with handle
validation error.

Reviewed by: kib
Reported by: venki kaps

Increase the size of the static TLS area slightly (required for the NVidia's
OpenGL driver on amd64).

Implement support for RTLD_NODELETE flag for dlopen() and -z nodelete
static linker option. Do it by incrementing reference count on the loaded
object and its dependencies.

Reviewed by: davidxu, kan

Implement the dynamic string token substitution in the rpath and
soneeded pathes. The $ORIGIN, $OSNAME, $OSREL and $PLATFORM tokens
are supported. Enabling the substitution requires DF_ORIGIN flag in
DT_FLAGS or DF_1_ORIGIN if DF_FLAGS_1, that may be set with -z origin
gnu ld flag. Translation is unconditionally disabled for setuid/setgid
processes.

The $ORIGIN translation relies on the AT_EXECPATH auxinfo supplied
by kernel.

Requested by: maho
Tested by: maho, pho
Reviewed by: kan

MFp4: Add mips support for dynamic linking.

This code came from the merged mips2 and Juniper mips repositories.
Warner Losh, Randall Seager, Oleksandr Tymoshenko and Olivier Houchard
worked to merge, debug and integrate this code. This code may also
contain code derived from NetBSD.

Implement dl_iterate_phdr function.
Convert boolean flags in internal Obj_Entry structure into bitfields.
Properly check for loaded segment alignment in map_object.

Implement ELF symbol versioning using GNU semantics. This code aims
to be compatible with symbol versioning support as implemented by
GNU libc and documented by http://people.redhat.com/~drepper/symbol-versioning
and LSB 3.0.

Implement dlvsym() function to allow lookups for a specific version of
a given symbol.

Make our ELF64 type definitions match standards. In particular this
means:
o Remove Elf64_Quarter,
o Redefine Elf64_Half to be 16-bit,
o Redefine Elf64_Word to be 32-bit,
o Add Elf64_Xword and Elf64_Sxword for 64-bit entities,
o Use Elf_Size in MI code to abstract the difference between
Elf32_Word and Elf64_Word.
o Add Elf_Ssize as the signed counterpart of Elf_Size.

MFC after: 2 weeks

Attempt to free any static TLS space used by a shared library when it
is unloaded. This allows applications which load and unload libraries
like libGL.so.1 several times to work properly.

MFC after: 2 days

Add support for Thread Local Storage.

Add initial support for compiling a special 32 bit version of
ld-elf.so.1 on 64 bit systems. Most of this involves using alternate
paths, environment variables and diagnostic messages.

The build glue is seperate.

Prepend /lib to the builtin library search path in rtld.

Provide a mechanism for dumping relocation information.

Setting the LD_DUMP_REL_PRE or LD_DUMP_REL_POST environment variables
cause rtld-elf to output a table of all relocations.

This is useful for debugging.

Move MD function prototypes together.

Fix warnings; no parameters in function prototypes.

- Add support for DT_FLAGS.
- Define various things from the most recent ELF spec.

Allow threading libraries to register their own locking
implementation in case default one provided by rtld is
not suitable.

Consolidate various identical MD lock implementation into
a single file using appropriate machine/atomic.h.

Approved by: re (scottl)

Implement dlinfo() function.

Introdice RTLD_SELF special handle and properly process it within
dlsym() and dlinfo() functions.

The intention is to improve our compatibility with Solaris and
to make a Java port easier.

Partially submitted by: phantom

Remove /usr/lib/elf from a default search path.
Move xprintf to malloc.c, it is only used there. Make static.

Submitted by: phantom

Update rtld for the "new" ia64 ABI. In the old toolchain, the
DT_INIT and DT_FINI tags pointed to fptr records. In 2.11.2, it points
to the actuall address of the function. On IA64 you cannot just take
an address of a function, store it in a function pointer variable and
call it.. the function pointers point to a fptr data block that has the
target gp and address in it. This is absolutely necessary for using
the in-tree binutils toolchain, but (unfortunately) will not work with
old shared libraries. Save your old ld-elf.so.1 if you want to use
old ones still. Do not mix-and-match.

This is a no-op change for i386 and alpha.

Reviewed by: dfr

Add ia64 support. Various adjustments were made to existing targets to
cope with a few interface changes required by the ia64. In particular,
function pointers on ia64 need special treatment in rtld.

Performance improvements for the ELF dynamic linker. These
particularly help programs which load many shared libraries with
a lot of relocations. Large C++ programs such as are found in KDE
are a prime example.

While relocating a shared object, maintain a vector of symbols
which have already been looked up, directly indexed by symbol
number. Typically, symbols which are referenced by a relocation
entry are referenced by many of them. This is the same optimization
I made to the a.out dynamic linker in 1995 (rtld.c revision 1.30).

Also, compare the first character of a sought-after symbol with its
symbol table entry before calling strcmp().

On a PII/400 these changes reduce the start-up time of a typical
KDE program from 833 msec (elapsed) to 370 msec.

MFC after: 5 days

* include/elf.h has been repo copied to include/elf-hints.h, and it no
longer includes machine/elf.h.
* consumers of elf.h now use the minimalist elf header possible.

This change is motivated by Binutils 2.11.0 and too much clashing over
our base elf headers and the Binutils elf headers.

Add support for dlsym(RTLD_DEFAULT, ...).

Revamp the code that calls shared libraries' init and fini functions.
Formerly the init functions were called in the opposite of the
order in which libraries were loaded, and libraries were loaded
according to a breadth-first traversal of the dependency graph.
That ordering came from SVR4.0, and it was easy to implement but
not always sensible.

Now we do a depth-first walk over the dependency graph and call
the init functions in an order such that each shared object's needed
objects are initialized before the shared object itself. At the
same time we build a list of finalization (fini) functions in the
opposite order, to guarantee correct C++ destructor ordering whenever
possible. (It may not be possible if dlopen and dlclose are used
in strange ways, but we come as close as one can come.)

The need for this renovation has become apparent as more programs
have started using multithreading. The multithreaded C library
libc_r requires initialization, whereas the standard libc does not.
Since virtually every other object depends on the C library, it is
important that it get initialized first.

Solve the dynamic linker's problems with multithreaded programs once
and for all (I hope). Packages such as wine, JDK, and linuxthreads
should no longer have any problems with re-entering the dynamic
linker.

This commit replaces the locking used in the dynamic linker with a
new spinlock-based reader/writer lock implementation. Brian
Fundakowski Feldman <green> argued for this from the very beginning,
but it took me a long time to come around to his point of view.
Spinlocks are the only kinds of locks that work with all thread
packages. But on uniprocessor systems they can be inefficient,
because while a contender for the lock is spinning the holder of the
lock cannot make any progress toward releasing it. To alleviate
this disadvantage I have borrowed a trick from Sleepycat's Berkeley
DB implementation. When spinning for a lock, the requester does a
nanosleep() call for 1 usec. each time around the loop. This will
generally yield the CPU to other threads, allowing the lock holder
to finish its business and release the lock. I chose 1 usec. as the
minimum sleep which would with reasonable certainty not be rounded
down to 0.

The formerly machine-independent file "lockdflt.c" has been moved
into the architecture-specific subdirectories by repository copy.
It now contains the machine-dependent spinlocking code. For the
spinlocks I used the very nifty "simple, non-scalable reader-preference
lock" which I found at

<http://www.cs.rochester.edu/u/scott/synchronization/pseudocode/rw.html>

on all CPUs except the 80386 (the specific CPU model, not the
architecture). The 80386 CPU doesn't support the necessary "cmpxchg"
instruction, so on that CPU a simple exclusive test-and-set lock
is used instead. 80386 CPUs are detected at initialization time by
trying to execute "cmpxchg" and catching the resulting SIGILL
signal.

To reduce contention for the locks, I have revamped a couple of
key data structures, permitting all common operations to be done
under non-exclusive (reader) locking. The only operations that
require exclusive locking now are the rare intrusive operations
such as dlopen() and dlclose().

The dllockinit() interface is now deprecated. It still exists,
but only as a do-nothing stub. I plan to remove it as soon as is
reasonably possible. (From the very beginning it was clearly
labeled as experimental and subject to change.) As far as I know,
only the linuxthreads port uses dllockinit(). This interface turned
out to have several problems. As one example, when the dynamic
linker called a client-supplied locking function, that function
sometimes needed lazy binding, causing re-entry into the dynamic
linker and a big looping mess. And in any case, it turned out to be
too burdensome to require threads packages to register themselves
with the dynamic linker.

Back out the previous change to the queue(3) interface.
It was not discussed and should probably not happen.

Requested by: msmith and others

Change the way that the queue(3) structures are declared; don't assume that
the type argument to *_HEAD and *_ENTRY is a struct.

Suggested by: phk
Reviewed by: phk
Approved by: mdodd

When a threads package registers locking methods with dllockinit(),
figure out which shared object(s) contain the the locking methods
and fully bind those objects as if they had been loaded with
LD_BIND_NOW=1. The goal is to keep the locking methods from
requiring any lazy binding. Otherwise infinite recursion occurs
in _rtld_bind.

This fixes the infinite recursion problem in the linuxthreads port.

Revamp the mechanism for enumerating and calling shared objects'
init and fini functions. Now the code is very careful to hold no
locks when calling these functions. Thus the dynamic linker cannot
be re-entered with a lock already held.

Remove the tolerance for recursive locking that I added in revision
1.2 of dllockinit.c. Recursive locking shouldn't happen any more.

Mozilla and JDK users: I'd appreciate confirmation that things still
work right (or at least the same) with these changes.

Add a new function dllockinit() for registering thread locking
functions to be used by the dynamic linker. This can be called by
threads packages at start-up time. I will add the call to libc_r
soon.

Also add a default locking method that is used up until dllockinit()
is called. The default method works by blocking SIGVTALRM, SIGPROF,
and SIGALRM in critical sections. It is based on the observation
that most user-space threads packages implement thread preemption
with one of these signals (usually SIGVTALRM).

The dynamic linker has never been reentrant, but it became less
reentrant in revision 1.34 of "src/libexec/rtld-elf/rtld.c".
Starting with that revision, multiple threads each doing lazy
binding could interfere with each other. The usual symptom was
that a symbol was falsely reported as undefined at start-up time.
It was rare but not unseen. This commit fixes it.

Make jdk-1.1.8 work again. It turns out that some code inside
libjava peeks into the dynamic linker's private Obj_Entry structures.
My recent changes introduced some new members near the front of
the structures, causing libjava to get the wrong fields. This commit
moves the new members toward the end of the structure so that the
layout of the portion that is relevant to JDK remains the same as
before.

I will work with the JDK porting team to see if we can come up with
a less fragile way for them to do what they need to do. I understand
the current approach was necessary in order to work around some
limitations of the dynamic linker. Maybe it's not necessary any
more.

Get the actual pathname of the dynamic linker from the executable's
PT_INTERP program header entry, to ensure that gdb always finds
the right dynamic linker.

Use obj->relocbase to simplify a few calculations where appropriate.

When checking to see if a shared object is already loaded, look for
a device/inode match if no pathname match is found.

Revamp the symbol lookup algorithm to cope better with objects
loaded separately by dlopen that have global symbols with identical
names. Viewing each dlopened object as a DAG which is linked by its
DT_NEEDED entries in the dynamic table, the search order is as
follows:

* If the referencing object was linked with -Bsymbolic, search it
internally.
* Search all dlopened DAGs containing the referencing object.
* Search all objects loaded at program start up.
* Search all objects which were dlopened() using the RTLD_GLOBAL
flag (which is now supported too).

The search terminates as soon as a strong definition is found.
Lacking that, the first weak definition is used.

These rules match those of Solaris, as best I could determine them
from its vague manual pages and the results of experiments I performed.

PR: misc/12438

$Id$ -> $FreeBSD$

Change many asserts into normal errors. They were all for conditions
caused by invalid shared objects rather than by internal errors.

Enable format string mismatch checking for _rtld_error().

Eliminate all machine-dependent code from the main source body and
the Makefile, and move it down into the architecture-specific
subdirectories.

Eliminate an asm() statement for the i386.

Make the dynamic linker work if it is built as an executable instead
of as a shared library. See i386/Makefile.inc to find out how to
do it. Note, this change is not enabled and it might never be
enabled. But it might be useful in the future. Building the
dynamic linker as an executable should make it start up faster,
because it won't have any relocations. But in practice I suspect
the difference is negligible.

Add alpha support.

Submitted by: John Birrell <jb@cimlogic.com.au> (with extra hacks by me)
Obtained from: Probably NetBSD

Suppress duplicate entries in ldd output.

Update this header to use the revamped elf headers which select Elf32
or Elf64 based on the inclusion of the machine dependent header.

I've left the addition of the extra fields to handle the relocation
structures with addend for a separate commit after jdp has had a chance
to review what I've done. The current change is needed to compile
csu/alpha/crt1.c

Add GDB support. The method and some of the code came from NetBSD's elf
runtime linker.

This commit was generated by cvs2svn to compensate for changes in r34192,
which included commits to RCS files with non-trunk default branches.

Import the ELF dynamic linker. This is the ElfKit version with
quite a few enhancements and bug fixes. There are still some known
deficiencies, but it should be adequate to get us started with ELF.

Submitted by: John Polstra <jdp@polstra.com>