rtld: Move powerpc specific code to powerpc files

There are two variables set by dynamic tags in the powerpc runtime
linker. Now we have a way to split out architecture-specific dynamic
tags use it to handle these.

Reviewed by: kib, jhibbits
Obtained from: jhibbits (earlier version)
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45182

rtld: Add arch_digest_dynamic

This will be used to handle the DT_AARCH64_VARIANT_PCS tag.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45117

rtld direct exec: make -u behavior match the description

Instead of only ignoring insecure env vars, clear them all.

Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44999

rtld: add direct-exec option -o

allowing to set any known LD_ parameter for the current rtld invocation,
but without polluting the activated' binary environment. In other
words, the set parameter is not exported into the environment.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44988

Support BTI in rtld

Read the elf note to decide when to set the guard page on arm64.

Reviewed by: kib
Sponsored by: Arm Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39452

rtld: reduce debug messages after fix on big-endian hosts

Remove a debug message that had been added to support the debugging
of a mis-detection of the hint files endianness on powerpc64.

MFC after: 3 days

rtld: fix check for endianess of elf hints file

Don't check if the elf hints file is in host byte order, but check
if it is in little endian by looking at the magic number.
This fixes rtld on big endian platforms.
Reviewed by: se, kib (prior version of the patch)
Fixes: 7b77d37a561b ("rtld-elf: support either byte-order of hints")
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D44472

rtld-elf: add some debug print statements

The byte-order independent code has been reported to fail on powerpc64.
Add some more debug statements to help identify the parametrs used and
to verify the correct operation of the byte-swap macros used..

rtld-elf: support either byte-order of hints file

Accept either little-endian or big-endian representation of the ELF
hints magic number in the header of a hints file and convert the
parameters to the native byte-order of the repsective system.

This is a pre-requisite for a planned change to always write the byte
order in little-endian format on all architectures. The only relvant
architecture that uses big-endian data is powerpc64, and it is not
likely that new architectures will choose that representation of data
in memory.

When all supported architectures use little-endian data in the hints
file, the byte swap logic can be enabled for big-endian CPUs at
compile time. Up to that point, there is a very small run-time penalty
that is paid on all systems to check the byte-order of the hints file
and to provide the option to byte-swap the parameters read from the
hints file header.

This commit contains the changes from review D44080 (which had been
split off from this patch for easier review),

Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D44053

fdlopen(3): do not create a new object mapping if already loaded

This is expected behavior for both dlopen(3) and fdlopen(3).

PR: 277169
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44019

rtld: ignore load_filtees() calls if we already loading filtees for the obj

in addition to avoiding it for already loaded filtees. Issue is that
during load, rtld needs to resolve some special ABI symbols, like
executable stack fixer and static TLS initializer, which might trigger
recursion.

Example is libthr which is filter for libsys, and which exports
__pthread_distribute_static_tls.

Tested by: kevans, krion
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43858

rtld load_filtees(): reindent and reduce block nesting

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43858

rtld symlook_obj: move common code to check filtees into helper

Revieved by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43858

rtld-elf: Avoid unnecessary lock_restart_for_upgrade() calls

In order to atomically upgrade the rtld bind lock, load_filtees() may
trigger a longjmp back to _rtld_bind() so that the binding can be done
with the write lock held. However, the write lock is only needed when
filtee objects haven't already been loaded, so move the
lock_restart_for_upgrade() call to avoid unnecessary lock upgrades when
a filtee is defined.

Reviewed by: kib
Tested by: brooks
MFC after: 1 week
Sponsored by: Innovate UK

RTLD_DEEPBIND: make lookup not just symbolic, but walk all refobj' DAGs

before starting the walk over the global list. Effectively we visit
needed objects first as well, instead of just the object itself.
This seems to better match the semantic offered by the glibc flag.

Reported by: kevans
PR: 275393
Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D42841

libexec: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

rtld: introduce STATIC_TLS_EXTRA

The new STATIC_TLS_EXTRA variable provides a means for applications
to increases the size of the extra static TLS space allocated by
rtld beyond the default of '128'. This extra static TLS space is used
for objects loaded with dlopen.

The value specified in the variable must be no less than the default
value and no greater than the maximum allowed value for size_t type.

If an invalid value is specified, rtld will ignore it and just use
the default value.

The rtld(1) man page is updated to document this new option.

Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D42025

rtld: output rtld errors into the dbg channel

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: switch from malloc_aligned() to __crt_aligned_alloc()

Use regular free(), since it works now.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D41150

rtld: unlock bind lock when calling into crt __pthread_distribute_static_tls method

The method might require resolving and binding symbols, which means
recursing on the bind lock. It is safe to unlock the bind lock,
since we operate on the private object list, and user attempting to
unload an object from the list of not yet fully loaded objects caused
self-inflicted race.

It is similar to how we treat user' init/fini methods.

Reported by: stevek
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

rtld: fix dlopen() for an object that is already mapped but not yet initialized

For instance, dso might be mapped as needed but not yet initialized from
the other subtree of needed objects, while current object' constructor
does dlopen() for the dso. Right now rtld does relocations and other
processing based on the arrival of new objects in the global list, which
is not happens there. Directly check for the initialization state of
the object, for which we would return the handle.

One practical use case of this support is e.g. dlopen("libthr.so",
RTLD_NOLOAD) by libraries that are threading-aware but happy to live
with libc pthread shims if the program is not multithreaded.

Reviewed by: tijl
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

rtld-elf: Migrate from COMPAT_32BIT to generic COMPAT_libcompat/LIBCOMPAT

We still have a tiny amount of libcompat-specific code in rtld_paths.h,
but it's been deduplicated as much as possible, and in future we may
wish to just push these variables down to the few consumers of them and
make them use the double-underscore variants with a libcompat argument
rather than give them names here.

See commit 8fad2cda93c7 ("bsd.compat.mk: Provide new CPP and sub-make
variables") for the context behind this change.

Reviewed by: kib, brooks, jhb
Differential Revision: https://reviews.freebsd.org/D40925

rtld-elf: Mark tls_init_align __unused in free_tls for Variant I TLS.

Some architectures (powerpc and RISC-V) always use 0 for the post TLS
size in which case tls_init_align isn't used by
calculate_tls_post_size. Use __unused to quiet the warning for these
platforms.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D40673

rtld: fix allocate_module_tls() variant I fallback to static allocation

Submitted by: Joerg Sonnenberger
Fixes: 91880e07f605edb90339685bc934699a4344de3bESC
MFC after: 1 week

rtld: do not allow both dynamic DTV index and static TLS offset

If we are allocating static offset for an object with dynamic index,
return failure. In the opposite case, if dynamic index is requested for
statically allocated TLS area, directly use the offset instead of
setting the index.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: rename tls_done to tls_static

The meaning of the flag is that static TLS allocation was done.

Taken from NetBSD Joerg Sonnenberger change for src/libexec/ld.elf_so/tls.c
rev. 1.18.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

rtld: fixes for handling of the grouped options

Do not terminate scanning group when 'd' is encountered.
The 'b' option must be last in the group, same as 'f'.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39544

rtld: fix SysV hash function overflow

Quoting from https://maskray.me/blog/2023-04-12-elf-hash-function:

The System V Application Binary Interface (generic ABI) specifies the
ELF object file format. When producing an output executable or shared
object needing a dynamic symbol table (.dynsym), a linker generates a
.hash section with type SHT_HASH to hold a symbol hash table. A DT_HASH
tag is produced to hold the address of .hash.

The function is supposed to return a value no larger than 0x0fffffff.
Unfortunately, there is a bug. When unsigned long consists of more than
32 bits, the return value may be larger than UINT32_MAX. For instance,
elf_hash((const unsigned char *)"\xff\x0f\x0f\x0f\x0f\x0f\x12") returns
0x100000002, which is clearly unintended, as the function should behave
the same way regardless of whether long represents a 32-bit integer or
a 64-bit integer.

Reviewed by: kib, Fangrui Song
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39517

rtld: reindent and style(9) elf_hash()

In preparation for an upcoming change (D39517).

Sponsored by: The FreeBSD Foundation

rtld: Revert "When loading dso without PT_GNU_STACK phdr, only call"

After the removal of ia64 and sparc64, all current architectures
support executable stacks at an architectural level.

This reverts commit 1290d38ac50b3afa7e5781d9d97346a1042c736c.

Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D37904

rtld: add support for the $LIB token

similar to the same token in glibc.

Requested and reviewed by: bapt
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D37329

rtld: remove unused macro FPTR_TARGET

It is a remnant from the ia64 removal

Sponsored by: The FreeBSD Foundation
MFC after: 3 days

ldd: remove '[preloaded]' marker for the preloaded objects

for the default output. For '-a' (per-object needed printout) the
[preloaded] banner is kept.

Instead, use special format2 for printing the preloaded objects (and
vdso), which does not include DT_NEEDED, since there is no object
needing the printed one.

In this way, the output is more compatible with glibc.

Example:
LD_PRELOAD=/lib/libthr.so.3 LD_TRACE_LOADED_OBJECTS=1 /libexec/ld-elf.so.1 /bin/ls
libutil.so.9 => /lib/libutil.so.9 (0x801099000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x8010b0000)
libc.so.7 => /lib/libc.so.7 (0x801123000)
[vdso] (0x7ffffffff000)
/lib/libthr.so.3 (0x80106c000)
Note the absense of the part before and including '=>' for preloaded
libthr.so.3, and for vdso.

PR: 265750
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D36616

rtld: teach LD_SHOW_AUXV about AT_USRSTACK*

Reviewed by: brooks, imp (previous version)
Discussed with: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D36540

rtld.c: remove extra blank line

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D36396

rtld: fix display of the mapbase for the traced objects

Commit 24d0c9c1f565bab6e introduced the following regression:
% ldd /bin/ls
/bin/ls:
libutil.so.9 => /lib/libutil.so.9 (0x1021000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x1021000)
libc.so.7 => /lib/libc.so.7 (0x1021000)
Note that the base address is the same for all displayed libraries.

Fix it by passing correct object to trace_print_obj().

Fixes: 24d0c9c1f565bab6e
Reviewed by: jrtc27
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D36198

rtld-elf: Fix leaks and wild frees in origin_subst

55abf23dd36b inverted the value passed to origin_subst_one when rolling
up the existing code into a loop. If the first token is found ($ORIGIN),
this results in a wild free of part of strtab. Processing the second
token works fine and will act how the first should have regardless of
whether found, allocating memory for the string without freeing.
Processing subsequent tokens however will then leak, regardless of
whether found, as they will also believe they need to allocate memory
and can't free the string.

Found by: CHERI
Reviewed by: kib, markj
Fixes: 55abf23dd36b ("rtld: make token substitution table-driven")
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D35792

Have rtld query the page size from the kernel

To allow for a dynamic page size on arm64 have the runtime linker
query the kernel for the currentl page size.

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34765

rtld: trace preloaded objects

Before, preloaded objects, if not listed as needed for any normally
linked objects, were silently ignored.

Preloaded objects are printed with the `[preloaded]` herald. The list
includes the objects not listed explicitly as recursive dependencies of
the main object, effectively dsos loaded by LD_PRELOAD mechanism.
vdso is listed as well, since it is not needed by anything.

Since there is no DT_NEEDED entry for LD_PRELOADed objects, they are
usually printed using LD_TRACE_LOADED_OBJECTS_FTM2 format due to the
failure of the heuristic based on the presence of the 'lib' prefix.

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34716

rtld trace_loaded_objects(): use bool for the list_containers variable

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34716

rtld: use style(9) for trace_loaded_objects()

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34716

rtld: extract printer for a single traced object into a helper

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34716

rtld: extract calculation of the format strings for trace into a helper

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34716

rtld: Add ${TOKEN} aliases to $TOKEN

it seems that glibc supports them, and such spelling is mentioned in the
ld.bfd manual. Idea seems to auto-correct some quoting/makefile sytnax
errors on linker command line.

Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34247

rtld: make token substitution table-driven

Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D34247

rtld: Fix assertion I broke.

textrel is a bool, not a pointer.

Pointy hat to: jhb
Reported by: kevans
Fixes: 6a2d75d24857 rtld: Trim a few more MIPS #ifdefs.

rtld: Trim a few more MIPS #ifdefs.

Reviewed by: imp, kib
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D34052

Restore DT_DEBUG processing

Pointed out by: kevans
Fixes: 292cba9b4919ee188ca08cc8bf5057e346514383
Sponsored by: The FreeBSD Foundation

rtld: remove mips-specific cases from generic code

Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D33763

TLS: Use <machine/tls.h> for libc and rtld.

- Include <machine/tls.h> in MD rtld_machdep.h headers.

- Remove local definitions of TLS_* constants from rtld_machdep.h
headers and libc using the values from <machine/tls.h> instead.

- Use _tcb_set() instead of inlined versions in MD
allocate_initial_tls() routines in rtld. The one exception is amd64
whose _tcb_set() invokes the amd64_set_fsbase ifunc. rtld cannot
use ifuncs, so amd64 inlines the logic to optionally write to fsbase
directly.

- Use _tcb_set() instead of _set_tp() in libc.

- Use '&_tcb_get()->tcb_dtv' instead of _get_tp() in both rtld and libc.
This permits removing _get_tp.c from rtld.

- Use TLS_TCB_SIZE and TLS_TCB_ALIGN with allocate_tls() in MD
allocate_initial_tls() routines in rtld.

Reviewed by: kib, jrtc27 (earlier version)
Differential Revision: https://reviews.freebsd.org/D33353

rtld: load preloaded vdso

Reviewed by: emaste
Discussed with: jrtc27
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D32960

Pass vdso address to userspace

Reviewed by: emaste
Discussed with: jrtc27
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D32960

rtld dump_auxv: be pedantic and distiguish between auxv union members based on format

Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld dump_auxv: consistently use long modifier for non-pointer auxv vals

Reviewed by: jrtc27
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Implement LD_SHOW_AUXV

It dumps auxv as seen by interpreter, right before starting any user
code.

Copied from: glibc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: style adjustments

Consistently use `return ()'.
Fix some spacing issues with types formatting, and around binary ops.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: ignore fstatfs(2) errors when checking MNT_NOEXEC flag

File descriptor we operate on might reference something that is not a
file, e.g. shmfd. In this case, we cannot check MNT_NOEXEC in
principle.

If fstatfs(2) caused some failure on normal filesystem, then typical
expectation is that read or mmap of this file would also fail. If not,
mmap(2) PROT_EXEC on MNT_NOEXEC filesystem returns EACCES.

Reported by: obiwac@gmail.com
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: fix dangerous_ld_env calculation for ld_dynamic_weak

Default value for ld_dynamic_weak is true, non-default settings should
be activated for the false value.

Reported by: obiwac@gmail.com
Sponsored by: The FreeBSD Foundation
MFC after: 3 days

rtld: Print currently configured search path for libraries for -v

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld-elf/paths.h: Make it usable outside rtld

but still for tightly coupled things like ldd(1)

Rename paths.h to rtld_paths.h.
Add guard for rtld-specific externs declarations.
Add _COMPAT32_BASENAME_RTLD and _COMPAT32_PATH_RTLD.

Reviewed by: arichardson, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32464

rtld: Support DT_RELR relative relocation format

PIE and shared objects usually have many relative relocations. In
2017/2018, a compact relative relocation format RELR was proposed on
https://groups.google.com/g/generic-abi/c/bX460iggiKg/m/GxjM0L-PBAAJ
("Proposal for a new section type SHT_RELR") and is a pre-standard.
RELR usually takes 3% or smaller space than R_*_RELATIVE relocations.
The virtual memory size of a mostly statically linked PIE is typically
5~10% smaller.

ld.lld --pack-dyn-relocs=relr emits RELR relocations. DT_RELR has been
adopted by Android bionic, Linux kernel's arm64 port, Chrome OS (patched
glibc).

This patch adds DT_RELR support to FreeBSD rtld-elf.

MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32524

rtld direct exec: add -d option

to ignore lack of execute permission on the binary. The check is a
bad security theatre anyway.

Reviewed by: arichardson, imp
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32464

rtld: constify most variables holding references to the environment values

Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31545

rtld: provide private getenv(3)

Reviewed by: arichardson, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31545

rtld: avoid use of of getenv(3) for evaluating rtld env vars (LD_XXX)

Scan through the set of environment variables during initialization and
store values in the corresponding ld_env_var_desc structure, in the
single pass at init time. This does not eliminate use of getenv(3) and
unsetenv(3) completely, but provides a foundation to do that as the next
step.

Also organize the scan in a way that makes it easier to support aliases
like LD_DEBUG vs. LD_64_DEBUG.

Suggested by: arichardson
Reviewed by: arichardson, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31545

rtld: rework how environment variables are named

Instead of specifying the main name part of the environment variable as the
string literal, create array of the var names and access them by symbolic
index. Convert main name parts into complete names by prefixing with
ABI-specific ld_env_vars.

This way the name is not repeated, and also it can carry additional
proporties explicitly. For instance, cleanup of the environment for
the setuid image does not require retyping all names.

Reviewed by: arichardson, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D31545

rtld: Switch to the standard symbol lookup behavior if LD_DYNAMIC_WEAK is set

The current lookup prefers a strong definition to a STB_WEAK definition
(similar to glibc pre-2.2 behavior) which does not conform to the ELF
specification.

The non-compliant behavior provoked https://reviews.llvm.org/D4418
which was intended to fix -shared-libasan but introduced
new problems (and caused some sanitizer tests (e.g.
test/asan/TestCases/interception_failure_test.cpp) to fail): sanitizer
interceptors are STB_GLOBAL instead of STB_WEAK, so defining a second
STB_GLOBAL interceptor can lead to a multiple definition linker error.
For example, in a -fsanitize={address,memory,...} build, libc functions
like malloc/free/strtol/... cannot be provided by user object files.

See
https://docs.freebsd.org/cgi/getmsg.cgi?fetch=16483939+0+archive/2014/freebsd-current/20140716.freebsd-current
for discussions.

This patch implements the ELF-compliant behavior when LD_DYNAMIC_WEAK is
set. STB_WEAK wrestling in symbol lookups in `Search the dynamic linker
itself` are untouched.

Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D26352

rtld: Remove calculate_tls_end

Variant I architectures use off and Variant II ones use size + off.
Define TLS_VARIANT_I/TLS_VARIANT_II symbols similarly to how libc
handles it.

Reviewed by: kib
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31539
Differential revision: https://reviews.freebsd.org/D31541

rtld: Round down relro_size

lld rounds up p_memsz(PT_GNU_RELRO) to satisfy common-page-size. If the
page size is smaller than common-page-size, rounding up relro_size may
incorrectly make some RW pages read-only.

GNU ld, gold, and ld.lld ensures p_vaddr+p_memsz is a multiple of
common-page-size. While max-page-size >= system the page size,
common-page-size can be smaller than the system page size.

Submitted by: MaskRay
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31498

rtld-elf: Check the return value of obj_enforce_relro()

The mprotect() call was failing on CheriBSD when changing rtld's relro
page permissions due to missing CHERI capability permissions on the
mprotect() argument but did not report an error since the return value
was being ignored. It should never fail on any supported FreeBSD
architecture, but checking the return value seems like a good
sanity check to me.

Reviewed By: kib, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30820

rtld: Rename -t option to -u (ignore LD_ vars)

Requested by: arichardson
Sponsored by: The FreeBSD Foundation
MFC after: 3 days

rtld direct exec: add option to ignore LD_ variables

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: preserve the 'seen' state of the dlerror message in errmsg_save()

rtld preserves its current error message around calls to user init/fini
lists, to not override original error with potential secondary errors
caused by user code recursing into rtld. After 4d9128da54f8f8e2a29190,
the preservation of the string itself is not enough, the 'seen'
indicator must be preserved as well. Otherwise, since new code does not
clear string (it cannot), call to _rtld_error() from errmsg_restore()
revived whatever message was consumed last.

Change errmsg_save() to return structure recording both 'seen' indicator
and the message, if any.

PR: 255698
Reported by: Eugene M. Kim <astralblue@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 3 days

rtld: make dlerror() thread-local

PR: 95339
Discussed with: arichardson
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29633

rtld: unstaticise lockinfo and obj_from_addr()

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29633

rtld: avoid recursing on rtld_bind_lock for write

This fixes a regression in d36d6816151705907393889, where the call to
__tls_get_address() was performed under rtld_bind_lock write-locked.
Instead use tls_get_addr_slow() directly, with locked = true.

Reported by: jkim, many others
Tested by: jkim, bdragon (powerpc), mhorne (riscv)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29623

rtld: allow to use tls_get_addr_slow() from context where rtld_bind_lock is locked

Explicit locked parameter is added

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29623

rtld: style tls_get_addr_slow

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29623

rtld dl_iterate_phdr(): dlpi_tls_data is wrong

dl_iterate_phdr() dlpi_tls_data should provide the TLS module segment
address, and not the TLS init segment address as it does now.

Reported by: emacsray@gmail.com
PR: 254774
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: introduce PRELOAD_FDS

The new PRELOAD_FDS variable accepts a list of file descriptors
that should be loaded into the process.

This may be used to optimize a loading process - in the case when
we already have a file descriptor to the library; we don't have
to look into multiple PATH to find it.

It may also be used in capability mode to load a single additional
library without the need to open a directory that contains it.

The last use of this functionality t may be a race-free method
of loading libraries.

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D29334

rtld: style nits

No functional change intended.

Requested by: kib

rtld: Fix null-pointer dereference

When a library is opened via fdlopen, it has a null pointer for its path
and so _rtld_bind can crash as a result of passing the null pointer to
basename() (which passes it to strrchr(), which doesn't do a null check).

PR: 253081
Submitted by: theraven
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28442

rtld: map without PROT_EXEC for relocation

This makes text segment relocation work under W^X.

Submitted by: Greg V <greg@unrelenting.technology> (original version)
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D27953

rtld-elf(1): remove obsolete pre_init() hook

It's no longer used since 600ee699ed2805894f5972c6ac2c3d17dca7f6ce
and r358358 respectively.

rtld: bump r_debug.r_version to 1 from current 0.

Add r_ldbase.

Requested and reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27429

rtld: pass argc/argv/env to dso inits.

This is consistent with how array inits are called, and also makes us
more compatible with glibc environment.

Requested by: Alex S <iwtcex@gmail.com>
PR: 249162
Reviewed by: dim, emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D26351

rtld: do not refuse to relocate objects without dynamic symtabs.

Such objects can still have valid relocations not requiring symbolic
references.

PR: 249121
Reported by: wsh@riski.sh
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: do not process absent dynamic.

If object has no dynamic phdr, do not try to dereference NULL. This
means that we cannot process any relocation, and that there cannot be
symbols defined, but it is up to static linker to produce meaningful
objects.

PR: 249121
Reported by: wsh@riski.sh
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Include FreeBSD ABI tag note in the ELF runtime loader.

Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D25306

Handle non-PLT GNU IFUNC relocations in rtld

In the last IFUNC related changes to rtld, the code that handled non-PLT
GNU IFUNC relocations ended up getting lost. This could leave some
relocations unhandled, causing crashes or misbehavior. This change restores
the handling of these relocations, but now together with the other IFUNC
relocations, allowing resolvers to reference external symbols.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Eldorado Research Institute (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D25550

rtld: Apply relro to itself.

Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D25319

rtld: Parse own phdr and notes.

Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D25319

rtld: Allow to load ET_DYN && DF_1_PIE when tracing.

This makes old ldd to still work on newer tagged PIE binaries.

Also move debug line for hashes before both decisions to not load are
done, so that the end of digest_dynamic() processing and reason to not
load or load is seen in debug trace.

Noted by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Add debug line for dlopen_object().

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Systematically pass RTLD_LO_TRACE to load_needed_objects().

Which makes all calls to load_object() to observe the flag, except the
calls for preloaded DSOs.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: set osrel when in the direct exec mode.

Rtld itself is a shared object which does not have vendor note, so
after the direct exec of ld-elf.so.1 process has p_osrel set to zero.
This affects the ABI of syscalls.

Set osrel to the __FreeBSD_version value at compile time right after
rtld identified direct exec mode. Then, switch to the osrel read from
the binary note or zero if no note, right before starting calling
ifunc resolvers, which is the first byte of the user code.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Uppercase 'dso' to indicate that it is abbreviation.

Suggested by: arichardson
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Do not allow to load ET_DYN object with DF_1_PIE flag set.

Linkers are supposed to mark PIE binaries with DF_1_PIE, such binary
cannot be correctly and usefully loaded neither by dlopen(3) nor as a
dependency of other object. For instance, we cannot do anything
useful with COPY relocations, among other things.

Glibc already added similar restriction.

Requested and reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D25086

rtld: Add -b option to allow to specify image name different from arg0.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Fix indent in print_usage().

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Add -v switch to print some useful information about the rtld binary.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Add -p switch to direct exec mode summary line.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Add version indicators to rtld.

It is wrong to relate on __FreeBSD_version, either from
include/param.h, kernel, or libc, to check for rtld features.
Rtld might be from newer world than the running userspace.

Add special private symbols exported by rtld itself, to indicate the
changes in runtime behavior, and features that cannot be otherwise
detected or deduced at runtime.

Note that the symbols are not exported from libc, so they intentionally
cannot be linked against, and exported from the private namespace from rtld.
Consumers are required to use dlsym(3). For instance, for
_rtld_version_laddr_offset, user should do
ptr = dlsym(RTLD_DEFAULT, "_rtld_version_laddr_offset")
or even
ptr = dlvsym(RTLD_DEFAULT, "_rtld_version_laddr_offset",
"FBSDprivate_1.0");
Non-null ptr means that the change is present.

Also add _rtld_version__FreeBSD_version indicator to report the
headers version used at time of the rtld build.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24982

Implement Solaris-like link_map l_refname member.

The implementation is based on the public documentation, in particular
dlinfo(3) from Solaris.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Convert linkmap_add() and linkmap_delete() to style(8).

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Restore the binary compatibility for link_map l_addr.

Keep link_map l_addr binary layout compatible, rename l_addr to l_base
where rtld returns map base. Provide relocbase in newly added l_addr.

This effectively reverts the patch to the initial version of D24918.

Reported by: antoine (portmgr)
Reviewed by: jhb, markj
Tested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24946

Change the samantic of struct link_map l_addr member.

It previously returned the object map base address, while all other
ELF operating systems return load offset, i.e. the difference between
map base and the link base.

Explain the meaning of the field in the man page.

Stop filling the mips-only l_offs member, which is apparently unused.

PR: 246561
Requested by: Damjan Jovanovic <damjan.jov@gmail.com>
Reviewed by: emaste, jhb, cem (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24918

Implement RTLD_DEEPBIND.

PR: 246462
Tested by: Martin Birgmeier <d8zNeCFG@aon.at>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24841

rtld: ignore static TLS segments when tracing.

For PIE binaries, ldd(1) performs dlopen(RTLD_TRACE) on the binary.
It is legal for binary to use initial exec TLS mode, but when such
binary (actually dso) is dlopened, we might not have enough free space
in the finalized static TLS segment. Make ldd operational by skipping
TLS space allocation, we are not going to execute any code from the
dso anyway.

Reported by: tobik
PR: 245677
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Align initial-exec TLS segments to the p_vaddr % align.

This is continuation of D21163/r359634, which handled the alignment
for global mode.

Non-x86 arches are not handled, maintainers are welcomed.

Tested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D24366

Make p_vaddr % p_align == p_offset % p_align for (some) TLS segments.

See https://sourceware.org/bugzilla/show_bug.cgi?id=24606 for the test case.
See https://reviews.llvm.org/D64930 for the background and more discussion.

Also this fixes another bug in malloc_aligned() where total size of
the allocated memory might be not enough to fit the aligned requested
block after the initial pointer is incremented by the pointer size.

Reviewed by: bdragon
Tested by: antoine (exp-run PR 244866), bdragon, emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D21163

Remove sparc64 specific parts of rtld-elf.

Handle non-plt IRELATIVE relocations, at least for x86.

lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by: dim (amd64, previous version)
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23652

Fix indent.

Sponsored by: The FreeBSD Foundation
MFC after: 3 days

Use sigfastblock(2) in rtld.

This allows for rtld to not issue two sigprocmask(2) syscalls for each
symbol binding operation in single-threaded processes. Rtld needs to
block signals as part of locking to ensure signal safety of the bind
process, because signal handlers might need to lazily resolve symbol
references.

As result, number of syscalls issued on startup by simple programs not
using libthr, is typically reduced 2x. For instance, for hello world,
I see:
non-sigfastblock
# (truss ./hello > /dev/null) |& wc -l
63
sigfastblock
# (truss ./hello > /dev/null) |& wc -l
37

Tested by: pho
Disscussed with: cem, emaste, jilles
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D12773

rtld: remove hand rolled memset and bzero

They were introduced to take care of ifunc, but right now no architecture
provides ifunc'ed variants. Since rtld uses memset extensively this results in
a pessmization. Should someone want to use ifunc here they should provide a
mandatory symbol (e.g., rtld_memset).

See the review for profiling data.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D23176

Stop prepending prefix to the result of realpath(3).

The path is already absolute.

Noted and reviewed by: rstone
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23121

rtld: Return error if $ORIGIN for a dlopen-ed library cannot be resolved ...

instead of killing the process. The same behaviour of terminating
image activation if the $ORIGIN cannot be resolved for the main
object, is kept.

Reported by: Greg V <greg@unrelenting.technology>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23053

Resolve relative argv0 for direct exec mode to absolute path for AT_EXECPATH.

We know the binary relative name and can reliably calculate cwd path.
Because realpath(3) was already linked into ld-elf.so.1, reuse it
there to resolve dots and dotdots making the path more canonical.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23014

rtld: fix after r356300

binpath local was changed from char array to a char pointer, update
strlcpy/strlcat uses.

Reported by: Coverity through vangyzen
CID: 1412239 and 1412240
Reviewed by: emaste, imp, vangyzen
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23090

rtld: Fix segfault in direct exec mode

When rtld is directly executed with arguments, it has to move the
program arguments, environment and elf aux data up a few slots to
remove its own arguments before the process being executed sees
them. When copying the environment, rtld was incorrectly testing
whether the location about to be written to currently contained
NULL, when was supposed to check whether it had just copied the
NULL terminator of the environment string. This had the result
that the ELF aux data was mostly treated as environment variables,
and rtld would quickly crash when it tried to access required
ELF aux data that it didn't think was present.

Differential Revision: https://reviews.freebsd.org/D23008
Reviewed by: kib
MFC after: 1 month

Fix AT_EXECPATH for direct exec mode.

When activated in direct exec mode, kernel-provided AT_EXECPATH points
to the interpreter. We need to recalculate auxv to point to the
string with the path to the executable which is actually executed.

The somewhat problematic case is when the executable path is relative
and either $PATH use is not enabled or it contains '/' so $PATH search
is not performed. In this case resulting AT_EXECPATH is relative, I
might fix this later.

Reported and reviewed by: rstone
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D22894

Eliminate the last MI difference in AT_* definitions (for powerpc).

Summary:
As a transition aide, implement an alternative elfN_freebsd_fixup which
is called for old powerpc binaries. Similarly, add a translation to rtld to
convert old values to new ones (as expected by a new rtld).

Translation of old<->new values is incomplete, but sufficient to allow an
installworld of a new userspace from an old one when a new kernel is running.

Test Plan:
Someone needs to see how a new kernel/rtld/libc works with an old
binary. If if works we can probalby ship this. If not we probalby need
some more compat bits.

Submitted by: brooks
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D20799

Fix aux_info corruption in rtld direct execution mode.

After the aux vector is moved, it is necessary to re-digest aux_info so the
pointers are updated to the new locations.

This was causing thread creation to fail on powerpc64 when using direct
execution due to a nonsense value being read for aux_info[AT_STACKPROT].

Approved by: jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D21656

Reduce size of rtld by 22% by pulling in less code from libc

Currently RTLD is linked against libc_nossp_pic which means that any libc
symbol used in rtld can pull in a lot of depedencies. This was causing
symbol such as __libc_interposing and all the pthread stubs to be included
in RTLD even though they are not required. It turns out most of these
dependencies can easily be avoided by providing overrides inside of rtld.

This change is motivated by CHERI, where we have an experimental ABI that
requires additional relocation processing to allow the use of function
pointers inside of rtld. Instead of adding this self-relocation code to
RTLD I attempted to remove most function pointers from RTLD and discovered
that most of them came from the libc dependencies instead of being actually
used inside rtld.

A nice side-effect of this change is that rtld is now 22% smaller on amd64.

text data bss dec hex filename
0x21eb6 0xce0 0xe60 145910 239f6 /home/alr48/ld-elf-x86.before.so.1
0x1a6ed 0x728 0xdd8 113645 1bbed /home/alr48/ld-elf-x86.after.so.1

The number of R_X86_64_RELATIVE relocations that need to be processed on
startup has also gone down from 368 to 187 (almost 50% less).

Reviewed By: kib
Differential Revision: https://reviews.freebsd.org/D20663

powerpc: Transition to Secure-PLT, like most other OSs

Summary:
PowerPC has two PLT models: BSS-PLT and Secure-PLT. BSS-PLT uses runtime
code generation to generate the PLT stubs. Secure-PLT was introduced with
GCC 4.1 and Binutils 2.17 (base has GCC 4.2.1 and Binutils 2.17), and is a
more secure PLT format, using a read-only linkage table, with the dynamic
linker populating a non-executable index table.

This is the libc, rtld, and kernel support only. The toolchain and build
parts will be updated separately.

Reviewed By: nwhitehorn, bdragon, pfg
Differential Revision: https://reviews.freebsd.org/D20598
MFC after: 1 month

Switch to use shared vnode locks for text files during image activation.

kern_execve() locks text vnode exclusive to be able to set and clear
VV_TEXT flag. VV_TEXT is mutually exclusive with the v_writecount > 0
condition.

The change removes VV_TEXT, replacing it with the condition
v_writecount <= -1, and puts v_writecount under the vnode interlock.
Each text reference decrements v_writecount. To clear the text
reference when the segment is unmapped, it is recorded in the
vm_map_entry backed by the text file as MAP_ENTRY_VN_TEXT flag, and
v_writecount is incremented on the map entry removal

The operations like VOP_ADD_WRITECOUNT() and VOP_SET_TEXT() check that
v_writecount does not contradict the desired change. vn_writecheck()
is now racy and its use was eliminated everywhere except access.
Atomic check for writeability and increment of v_writecount is
performed by the VOP. vn_truncate() now increments v_writecount
around VOP_SETATTR() call, lack of which is arguably a bug on its own.

nullfs bypasses v_writecount to the lower vnode always, so nullfs
vnode has its own v_writecount correct, and lower vnode gets all
references, since object->handle is always lower vnode.

On the text vnode' vm object dealloc, the v_writecount value is reset
to zero, and deadfs vop_unset_text short-circuit the operation.
Reclamation of lowervp always reclaims all nullfs vnodes referencing
lowervp first, so no stray references are left.

Reviewed by: markj, trasz
Tested by: mjg, pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D19923

Fix order of destructors between main binary and libraries.

Since inits for the main binary are run from rtld (for some time), the
rtld_exit atexit(3) handler, which is passed from rtld to the program
entry and installed by csu, is installed after any atexit(3) handlers
installed by main binary constructors. This means that rtld_exit() is
fired before main binary handlers.

Typical C++ static constructors are executed from init (either binary
or libs) but use atexit(3) to ensure that destructors are called in
the right order, independent of the linking order. Also, C++
libraries finalizers call __cxa_finalize(3) to flush library'
atexit(3) entries. Since atexit(3) entry is cleared after being run,
this would be mostly innocent, except that, atexit(rtld_exit) done
after main binary constructors, makes destructors from libraries
executed before destructors for main.

Fix by reordering atexit(rtld_exit) before inits for main binary, same
as it happened when inits were called by csu. Do it using new private
libc symbol with pre-defined ABI.

Reported. tested, and reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Fix initial exec TLS mode for dynamically loaded shared objects.

If dso uses initial exec TLS mode, rtld tries to allocate TLS in
static space. If there is no space left, the dlopen(3) fails. If space
if allocated, initial content from PT_TLS segment is distributed to
all threads' pcbs, which was missed and caused un-initialized TLS
segment for such dso after dlopen(3).

The mode is auto-detected either due to the relocation used, or if the
DF_STATIC_TLS dynamic flag is set. In the later case, the TLS segment
is tried to allocate earlier, which increases chance of the dlopen(3)
to succeed. LLD was recently fixed to properly emit the flag, ld.bdf
did it always.

Initial test by: dumbbell
Tested by: emaste (amd64), ian (arm)
Tested by: Gerald Aryeetey <aryeeteygerald_rogers.com> (arm64)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D19072

rtld: disable relro enforcement for irelative relocation processing.

This fixes yet another breakage for relro + bind now.

Reported by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 3 days

Adjust posix symbols from rtld-elf/malloc.c with the __crt_ prefix.

This allows to reuse the allocator in other environments that get
malloc(3) and related functions from libc or interposer.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18988

Remove now redundand ifunc relocation code which should have been
removed as part of r341441.

This call to reloc_non_plt() may crash if ifunc resolvers use the
needed libraries symbols since the pass over the needed libs
relocation is not yet done. The change in r341441 ensures the right
relocation order otherwise.

Submitted by: theraven
MFC after: 1 week
Discussed in: https://reviews.freebsd.org/D17529

Revert r343093 until I can address the issues raised by kib@.

Implement dlopenat(3).

MFC after: 3 weeks

Improve R_AARCH64_TLSDESC relocation.
The original code did not support dynamically loaded libraries and used
suboptimal access to TLS variables.
New implementation removes lazy resolving of TLS relocation - due to flaw
in TLSDESC design is impossible to switch resolver function at runtime
without expensive locking.

Due to this, 3 specialized resolvers are implemented:
- load time resolver for TLS relocation from libraries loaded with main
executable (thus with known TLS offset).
- resolver for undefined thread weak symbols.
- slower lazy resolver for dynamically loaded libraries with fast path for
already resolved symbols.

PR: 228892, 232149, 233204, 232311
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18417

Some fixes for LD_BIND_NOW + ifuncs.

- Do not perform ifunc relocations together with other PLT relocations
in PLT. Instead, do it during an additional pass over the init
list, so that ifuncs are resolved in the order of dso
dependencies. This allows the ifuncs resolvers to call into depended
libs. Init list now includes all objects instead of only objects
with init/fini callables.
- Disable relro protection around bind_now ifunc relocations.

I considered calling ifunc resolvers of dso after initializers of all
dependencies are processed, and decided that this is wrong/should not
be supported. The order now is normal relocations for all
objects->ifunc resolution in init order->initializers, where each step
does complete pass over all loaded objects before moving to the next
step.

Reported, tested and reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18400

Provide naive but self-contained implementations of memset(3) and
bzero(3) for rtld.

This again reduces rtld dependency on libc, and in future, avoid ifunc
relocations when the functions are converted to ifuncs in libc.

Reported by: mjg
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18400

rtld: parse FreeBSD Feature Control note on the object load.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

rtld: when immediate bind mode is requested, process irelocs in PLT
immediately after other PLT relocs.

Otherwise, if the object has relro page, we write to readonly page,
and we would need to use mprotect(2) two more times to fix it. Note
that resolve_object_ifunc() does nothing when called second time, so
there is no need to avoid existing call.

Reported and tested by: emaste
PR: 233333
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Handle the DT_MIPS_RLD_MAP_REL dynamic tag in RTLD

This dynamic tag contains the location of the .rld_map section relative to
the location of the dynamic tag. For PIE MIPS binaries DT_MIPS_RLD_MAP can
not be used since it contains an absolute address. Without this change
GDB can not find the function program counters in other libraries and once
I apply this change I can successfully run info sharedlibraries again.

Reviewed By: kib
Differential Revision: https://reviews.freebsd.org/D17867

rtld: move relro enforcement after ifunc processing

Previously the combination of relro (implicit), -z now and ifunc use
resulted in a segfault when applying ifuncs after relro (test binary
here just calls amd64_get_fsbase()):

| % env LD_DEBUG=1 libexec/rtld-elf/obj/ld-elf.so.1 a.out
| ...
| enforcing main obj relro
| ...
| resolving ifuncs
| reloc_jmpslot: *0x203198 = 0x189368ea4570
| zsh: bus error (core dumped) LD_DEBUG=1 obj/ld-elf.so.1 ~/a.out

Reported by: Shawn Webb
Reviewed by: kib
Sponsored by: The FreeBSD Foundation

Remove Obj_Entry textsize member.

It is unused after r340102, and more important, I do not see how to
define textsize in both practically useful and correct way, for binaries
with more that one executable segments.

Sponsored by: The FreeBSD Foundation

Initialize ifunc calling machinery earlier.

In particular, do it before the first call to allocate_initial_tls(),
which contains MD parts to set the initial thread' TLS pointer.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld-elf: fix more warnings to allow compiling with WARNS=6

Reviewed By: kib
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17154

rtld-elf: compile with WANRS=4 warnings other than -Wcast-align

Reviewed By: kib
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17153

rtld-elf: make it compile with WARNS=3

Reviewed By: kib
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17150

rtld: set obj->textsize correctly

With lld-generated binaries the first PT_LOAD will usually be a read-only
segment unless you pass --no-rosegment. For those binaries the textsize is
determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries to
be parsed correctly use the end of the last PT_LOAD that is marked as
executable instead.

I noticed that the value was wrong while adding some debug prints for some rtld
changes for CHERI binaries. `obj->textsize` only seems to be used by PPC so the
effect is untested. However, the value before was definitely wrong and the new
result matches the phdrs.

Reviewed By: kib
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17117

Provide refobj context when doing libmap substitution inside
search_library_path().

This corrects the scope of libmap matches.

Reported and tested by: Andreas Longwitz <longwitz@incore.de>
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
MFC after: 1 week

Rework rtld's TLS Variant I implementation to match r326794

The above commit fixed handling overaligned TLS segments in libc's
TLS Variant I implementation, but rtld provides its own implementation
for dynamically-linked executables which lacks these fixes. Thus,
port these changes to rtld.

This was previously commited as r337978 and reverted in r338149 due to
exposing a bug the ARM rtld. This bug was fixed in r338317 by mmel.

Submitted by: James Clarke
Approved by: re (kib)
Reviewed by: kbowling
Testing by: kbowling (powerpc64), br (riscv), kevans (armv7)
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16510

Revert r337978: Rework rtld's TLS Variant I implementation to match r326794

Michal Meloun reports that it breaks ctype (isspace()..) related
functions on armv7 so back out while we diagnose the issue.

Reported by: Michal Meloun <melounmichal@gmail.com>

Rework rtld's TLS Variant I implementation to match r326794

The above commit fixed handling overaligned TLS segments in libc's
TLS Variant I implementation, but rtld provides its own implementation
for dynamically-linked executables which lacks these fixes. Thus,
port these changes to rtld.

Submitted by: James Clarke
Reviewed by: kbowling
Testing byL kbowling (powerpc64), br (riscv), kevans (armv7)
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16510

Make sure the rtld(1) error messages go to stderr, not stdout.
While here fix capitalization of a few nearby strings, add the
rtld's file name prefix so it's obvious where the message come
from, and return zero when "-h" is used.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16530

Make rtld use libc_nossp_pic.a. Remove SSP shims.

Submitted by: Luis Pires
Reviewed by: brooks
Differential revision: https://reviews.freebsd.org/D15341

o Let rtld(1) set up psABI user trap handlers prior to executing the
objects' init functions instead of doing the setup via a constructor
in libc as the init functions may already depend on these handlers
to be in place. This gets us rid of:
- the undefined order in which libc constructors as __guard_setup()
and jemalloc_constructor() are executed WRT __sparc_utrap_setup(),
- the requirement to link libc last so __sparc_utrap_setup() gets
called prior to constructors in other libraries (see r122883).
For static binaries, crt1.o still sets up the user trap handlers.
o Move misplaced prototypes for MD functions in to the MD prototype
section of rtld.h.
o Sprinkle nitems().

libexec: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Remove unneeded calls to access(2) from rtld(1); just call open(2) instead.
The result looks like this:

--- przed 2017-10-21 23:19:21.445034000 +0100
+++ po 2017-10-21 23:18:50.031865000 +0100
@@ -11,7 +11,6 @@ mmap(0x0,102,PROT_READ,MAP_PRIVATE,3,0x0) = 343665418
close(3) = 0 (0x0)
open("/usr/local/etc/libmap.d",O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC,0165) ERR#2 'No such file or directory'
munmap(0x80067d000,102) = 0 (0x0)
-access("/usr/local/lib/libintl.so.8",F_OK) = 0 (0x0)
openat(AT_FDCWD,"/usr/local/lib/libintl.so.8",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=642560,size=55188,blksize=32768 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 34366541824 (0x80067d000)
@@ -20,14 +19,13 @@ mmap(0x800877000,40960,PROT_READ|PROT_EXEC,MAP_PRIVATE
mmap(0x800a81000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0xa000) = 34370752512 (0x800a81000)
munmap(0x80067d000,4096) = 0 (0x0)
close(3) = 0 (0x0)
-access("/usr/local/lib/libc.so.7",F_OK) ERR#2 'No such file or directory'
+openat(AT_FDCWD,"/usr/local/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,00) ERR#2 'No such file or directory'
openat(AT_FDCWD,"/var/run/ld-elf.so.hints",O_RDONLY|O_CLOEXEC,00) = 3 (0x3)
read(3,"Ehnt\^A\0\0\0\M^@\0\0\0\M-2\0\0"...,128) = 128 (0x80)
fstat(3,{ mode=-r--r--r-- ,inode=970684,size=306,blksize=32768 }) = 0 (0x0)
lseek(3,0x80,SEEK_SET) = 128 (0x80)
read(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,178) = 178 (0xb2)
close(3) = 0 (0x0)
-access("/lib/libc.so.7",F_OK) = 0 (0x0)
openat(AT_FDCWD,"/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=1605239,size=1910320,blksize=32768 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 34366541824 (0x80067d000)

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12766

Replace lseek(2)/read(2) pair with pread(2), removing yet another syscall
from the binary startup code.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL

Make find_library() conform to style(9). No functional changes.

Suggested by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL

Reword the conditional; it was ugly, and adding another parameter,
which I'm going to do in a subsequent commit, would make it even uglier.
No functional changes.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL

Handle relocations for newer non-PIC MIPS ABI.

Newer binutils supports extensions to the MIPS ABI for non-PIC code
that is used when compiling O32 binaries with clang 5 (but not used
for N64 oddly enough). These extensions require support for
R_MIPS_COPY relocations as well as a second PLT GOT using
R_MIPS_JUMP_SLOT relocations.

For R_MIPS_COPY, use the same approach as on other architectures where
fixups are deferred to the MD do_copy_relocations.

The additional PLT GOT for jump slots is located in a .got.plt section
which is identified by a DT_MIPS_PLTGOT dynamic entry. This GOT also
requires fixups for the first two GOT entries just as the normal GOT.
However, the entry point for this second GOT uses a different calling
convention. Rather than passing an offset into the GOT, it passes an
offset into the .rel.plt section. This requires a second entry point
(_rtld_pltbind_start) which calls the normal _rtld_bind() rather than
_mips_rtld_bind(). This also means providing a real version of
reloc_jmpslot() which is used by _rtld_bind().

In addition, add real implementions of reloc_plt() and
reloc_jmpslots() which walk .rel.plt handling R_MIPS_JUMP_SLOT
relocations.

Reviewed by: kib
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D12326

Read max_stack_flags from correct object.

'obj' is not initialized here.

Reviewed by: kib
MFC after: 1 week
Sponsored by: DARPA / AFRL

o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by: ngie
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D11901

Allow to specify targets by absolute paths in libmap.conf.

Submitted by: Tatu Kilappa <tatu.kilappa@iki.fi>
PR: 221032
MFC after: 2 weeks

In open_binary_fd: when using buffer size for strl* and snprintf,
always use >= instead of > to avoid truncation.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D11474
MFC after: 3 days

When reporting undefined symbol, note the version, if specified.

Use the standard syntax of name@version, I do not expect a confusion
due to unlikely possibility of the name containing the '@' character.

Requested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

For ld.so direct execution mode, implement -p option: search for the
binary in $PATH.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D10790

Update my copyright, note The FreeBSD Foundation involvement.
While tweaking copyright block, switch to use __FBSDID for tag.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Fix style [1], add static keyword before static function definition.

Noted by: bapt [1]
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Allow rtld direct-exec to take a file descriptor.

When executing rtld directly, allow a file descriptor to be explicitly
specified rather than opened from the given path. This, together with the
LD_LIBRARY_PATH_FDS environment variable, allows dynamically-linked
applications to be executed from within capability mode.

Also add some rudimentary argument parsing (without pulling in getopt or
the like) to accept this file descriptor, a help (-h) option and a basic
usage string.

Reviewed by: kib
Sponsored by: NSERC, RDC
Differential Revision: https://reviews.freebsd.org/D10751

Pretend that there is some security when executing in direct mode.

Do not allow direct exec if we the process is suid. Try to follow Unix
permission checks for DACs, ignore ACLs.

Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D10750

Rename rtld's parse_libdir to parse_integer.

This is a more accurate name, as the integer doesn't have to be a library
directory descriptor. It is also a prerequisite for more argument parsing
coming in the near future (e.g., parsing explicit binary descriptors).

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: NSERC

Make ld-elf.so.1 directly executable.

Check if passed phdr is actually phdr of the interpreter itself, and
decide that this is the case of direct execution. In this case, the
binary to activate is specified in the argv[1]. After opening it,
shift down on-stack structure with argv, env and aux vectors to
emulate execution of the binary and not of the interpreter.

Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D10701

In _rtld(), reorder local declarations to compact the block and
partially sort them by style(9). Move locals declarations from nested
blocks into the block at function start.

Discussed with: emaste
MFC after: 1 week

Disable LD_BIND_NOT for setugid processes.

Requested by: jilles
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Implement LD_BIND_NOT knob for rtld.

From the manpage:
When set to a nonempty string, prevents modifications of the PLT slots
when doing bindings. As result, each call of the PLT-resolved
function is resolved. In combination with debug output, this provides
complete account of all bind actions at runtime.

Same feature exists on Linux and Solaris.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Avoid bind lock recursion.

When dlclose(3) unloads an object with filtees, it recursively calls
dlclose(3) on each filtee in free_needed_filtees(). Introduce
dlclose_locked() helper, called from free_needed_filtees() instead of
dlclose(), and pass the bind lockstate down to avoid recursing.

Reported and tested by: jhibbits
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Handle protected symbols in rtld.

Protected symbol reference in GOT of the defining object must be
resolved to itself, same as -Bsymbolic globally.

Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D9317

Extend LD_UTRACE by also generating utrace(2) log events for runtime linker
errors.

Reviewed by: kib, jhb
Approved by: jhb(mentor)
MFC after: 1 week
Differential Revision: D9347

For the main binary, postpone enforcing relro read-only protection
until copy relocations are done.

Newer binutils and lld seems to output copy into relro-protected range.

Reported by: Rafael Espц╜ndola via emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

rtld: Fix a couple of bugs around the unloading of ELF filters.

- Pass the correct object to unload_filtees().
- Use a marker to restart iteration after unload_filtees() has returned.
It calls dlclose() and may recursively remove entries from the global
object list, so TAILQ_FOREACH_SAFE is not sufficient.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

rtld: Ensure that dlopen() cannot obtain a reference on a doomed object.

rtld drops the bind lock to call fini functions in an object prior to
unmapping it. The new "doomed" state flag prevents the acquisition of new
references for an object while the lock is dropped.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

rtld: Fix a race between dl_iterate_phdr() and dlclose().

Add a transient reference count to ensure that the phdr argument to the
callback remains valid while the bind lock is dropped.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Fix typo.

MFC after: 3 days

Take write lock for rtld_bind before modifying obj_list in dl_iterate_phdr().

This avoids a race with readers such as dladdr(3)/dlinfo(3)/dlsym(3) and
the atexit(3) handler. This race was introduced in r294373.

Reviewed by: markj, kib, kan
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

Retire long-broken/unused static rtld support

rtld-elf has some vestigial support for building as a static executable.
r45501 introduced a partial implementation with a prescient note that it
"might never be enabled." r153515 introduced ELF symbol versioning
support, and removed part of the unused build infrastructure for static
rtld.

GNU ld populates rela relocation addends and GOT entries with the same
values, and rtld's run-time dynamic executable check relied on this.
Alternate toolchains may not populate the GOT entries, which caused
RTLD_IS_DYNAMIC to return false. Simplify rtld by just removing the
unused check.

If we want to restore static rtld support later on we ought to introduce
a build-time #ifdef flag.

PR: 214972
Reviewed by: kan
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8687

Fix _mips_rtld_bind() to handle ELF filters.

MIPS does not use the common _rtld_bind() to handle runtime binding.
Instead, it uses a private _mips_rtld_bind(). Update _mips_rtld_bind()
to include the changes made to _rtld_bind() in r216695 and r218476 to
support upgrading the read-locked rtld_bind_lock to a write lock when
an object with a filter is encountered.

While here, add a 'where' variable to track the location of the fixup
in the GOT to make the code flow more closely match _rtld_bind().

Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D8625

Pass CPUID[1] %edx (cpu_feature), %ecx (cpu_feature2) and
CPUID[7].%ebx (cpu_stdext_feature), %ecx (cpu_stdext_feature2) to the
ifunc resolvers on x86.

It is much more clean to use CPUID instruction in usermode to retrieve
this information than to pass AT_HWCAP aux vector from kernel, on
x86. Still, the change does allow for use of AT_HWCAP on arches where it is
needed, by passing aux array to ifunc_init() initializer which should
prepare arguments for ifunc resolvers.

Current signature for resolvers on x86 is
func_t iresolve(uint32_t cpu_feature, uint32_t cpu_feature2,
uint32_t cpu_stdext_feature, uint32_t cpu_stdext_feature2);
where arguments have identical meaning as the kernel variables of the
same name. The ABIs allow to use resolvers with the void or shortened
list of arguments.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D8448

Pass the correct flag to find_symdef() from _rtld_bind().

When symbol versioning was added to rtld, the boolean 'in_plt' argument
to find_symdef() was converted to a bitmask of flags. The first flag
added was 'SYMLOOK_IN_PLT' which replaced the 'in_plt' bool. This
happened to still work by accident as SYMLOOK_IN_PLT had the value of 1
which is the same as 'true', so there should be no functional change.

Tested on: amd64
Reviewed by: kan
MFC after: 2 weeks
Sponsored by: DARPA / AFRL

Move defines common between rtld and libsysdecode into the header,
instead of copying inline into sources.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Fill phdr and phsize for rtld object. It is needed for
dl_iterate_phdr() reporting the correct values.

PR: 211367
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Remove all remaining uses of TAILQ_FOREACH_FROM() from rtld-elf.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Fix dlsym(RTLD_NEXT) handling to only return the next library in last library cases.

The root of the problem here is that TAILQ_FOREACH_FROM will default to
the head of the list if passed NULL, which will be the case if there are
no libraries loaded after this one. Thus all libraries, including the
current, were iterated in that case rather than none.

This was broken in r294373.

Reviewed by: markj (earlier version), cem, kib, ngie
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D7216

Fix issues found by Coverity in the rtld-elf.c:gethints().

Check that the dirlist path string specification does not cause
overflow and is fully contained in the hints file.
Check that the dirlist string is nul-terminated.
Make 'hdr' static variable, so that hdr.dirlistlen is available when
hints cached value is used on next function calls. Reset hdr.dirlistlen
to zero if error was detected, so that allocations use reasonable size.
Use 'hints', and not 'p' in the body, since p is only initialized on the
first call.

Reported and reviewed by: truckman (previous version)
Sponsored by: The FreeBSD Foundation
CIDs: 1006503, 1006504, 1006676, 1008488, 1007263
MFC after: 2 weeks

libexec: minor spelling fixes in comments.

No functional change.

rtld-elf: use NULL instead of zero for pointers.

Fix handling of DT_TEXTREL for an object with more than one read-only
segment. According to gABI spec, presence of the tag indicates that
dynamic linker must be prepared to handle relocations against any
read-only segment, not only the segment which we, somewhat arbitrary,
declared the text.

For each read-only segment, add write permission before relocs are
processed, and return to the mapping mode requested by the phdr, after
relocs are done.

Reported, tested, and reviewed by: emaste
PR: 207631
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Do not unlock rtld_phdr_lock over callback invocations.

The dl_iterate_phdr consumer code in libgcc does not expect multiple
callbacks running concurrently. This was fixed once already in r178807,
but accidentally got reverted in r294373.

Fix initlist_add_object invocation parameters.

The tail parameter should point to the last object for
which dependencies should be processed. In most cases,
this is the object itself.

Do not call callbacks for dl_iterate_phdr(3) with the rtld bind and
phdr locks locked. This allows to call rtld services from the
callback, which is only reasonable for dlopen(path, RTLD_NOLOAD) to
test existence of the library in the image, and for dlsym(). The
later might still be not quite safe, due to the lazy resolution of
filters.

To allow dropping the locks around iteration in dl_iterate_phdr(3), we
insert markers to track current position between relocks. The global
objects list is converted to tailq and all iterators skip markers,
globallist_next() and globallist_curr() helpers are added.

Reported and tested by: davide
Reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks

Restore ABI variants now that ldconfig groks -soft. In addition, as a
transition mechanism, if we don't have /usr/libsoft, assume that soft
float ABI binaries are the default, so treat them as default binaries.
When we've fully transitioned, it will make no sense to do this stat,
and it will be removed.

Revert r293201, r293202 (rtld: populate DT_DEBUG iff DYNAMIC segment is writable)

It turns out MIPS binaries may have other oddities that can trigger a
fault at startup.

PR: 206017
Reported by: ray

Disable abi variant hook until strangeness with packages can be sorted
out.

rtld: populate DT_DEBUG iff DYNAMIC segment is writable

MIPS has/had a read-only DYNAMIC segment, and uses an extra level of
indirection (through MIPS_RLD_MAP) to locate the debugger rendezvous
data.

Some linkers (e.g. LLVM's lld) may produce MIPS binaries with a writable
DYNAMIC segment, which would allow us to eventually drop a special case.

Therefore, instead of hardcoding knowledge that DYNAMIC is not writable
on MIPS just check the permissions on the segment.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4791

If md_exec_hook is defined, provide a way to create the strings
for the environment variables we look up at runtime. Otherwise,
there's no way they will change, optimize it at compile time.

Differential Review: https://reviews.freebsd.org/D2718

Create a generalized exec hook that different architectures can hook
into if they need to, but default to no action.

Differential Review: https://reviews.freebsd.org/D2718

Use a macro to create the names for the library path names. This will
allow later substitution at run time instead of compile time of the
environment variable name prefix.

Differential Review: https://reviews.freebsd.org/D2718

rtld: remove old XXX comment missed in r35529

Sponsored by: The FreeBSD Foundation

Add support for RISC-V architecture.

Reviewed by: andrew, kib
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
Differential Revision: https://reviews.freebsd.org/D4679

rtld: Use common NT_FREEBSD_* note types introduced in r291909

Sponsored by: The FreeBSD Foundation

Use variable names that aren't as prone to dyslexic confusion.

Suggested by: jmallet@

Provide support for ELFv2 userland if using a newer compiler (recent clang
or gcc) and binutils >= 2.24. Not enabled by default.

Rather than using the #define for path names, indirect through a char *
variable that could change for different executable types detected.

Move all the paths into a new path.h to centralize them.

Ensure we use calculate_first_tls_offset, even if the main program doesn't
have TLS program header. This is needed on architectures with Variant I
tls, that is arm, arm64, mips, and powerpc. These place the thread control
block at the start of the buffer and, without this, this data may be
trashed.

This appears to not be an issue on mips or powerpc as they include a second
adjustment to move the thread local data, however this is on arm64 (with a
future change to fix placing this data), and should be on arm. I am unable
to trigger this on arm, even after changing the code to move the data
around to make it more likely to be hit. This is most likely because my
tests didn't use the variable in offset 0.

Reviewed by: kib
MFC after: 1 week
Sponsored by: ABT Systems Ltd

Remove historical GNUC test

The requirement is for a GCC-compatible compiler and not necessarily
GCC itself. However, we currently expect any compiler used for building
the whole of FreeBSD to be GCC-compatible and many things will break if
not; there's no longer a need to have an explicit test for this in rtld.

Reviewed by: imp, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2422

Change interpretation of the DF_ORIGIN and DF_1_ORIGIN flags.
According to standard, the presence of the flags only means that the
object path must be resolved at the time object loading, instead of my
reading that the flag is required to enable token substitution at all.

The consequence is that -z origin linker flag is no longer required
for the token substitution in the run/rpath or the needed library
soname. It is only recommended if token substition is needed at
dlopen(3) time, since namecache might drop the required entries at the
time of resolution.

Found, reviewed and tested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Support file verification in MAC.

* Add VCREAT flag to indicate when a new file is being created
* Add VVERIFY to indicate verification is required
* Both VCREAT and VVERIFY are only passed on the MAC method vnode_check_open
and are removed from the accmode after
* Add O_VERIFY flag to rtld open of objects
* Add 'v' flag to __sflags to set O_VERIFY flag.

Submitted by: Steve Kiernan <stevek@juniper.net>
Obtained from: Juniper Networks, Inc.
GitHub Pull Request: https://github.com/freebsd/freebsd/pull/27
Relnotes: yes

Implement support -z global linker option. It marks the shared object
as always participating in the global symbols namespace, regardless of
the way the object was brought into the process address space.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Make die available as rtld_die for use by MD relocation code

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2213

Add the arm64 code to the runtime linker. It's not able to be built as we
still need libc_pic for a few things, but this is expected to be ready
soon.

Differential Revision: https://reviews.freebsd.org/D2136
Reviewed by: kib
Sponsored by: The FreeBSD Foundation

Change compiler setting to make default visibility of the symbols for
rtld on x86 to be hidden. This is a micro-optimization, which allows
intrinsic references inside rtld to be handled without indirection
through PLT. The visibility of rtld symbols for other objects in the
symbol namespace is controlled by a version script.

Reviewed by: kan, jilles
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

When a file is executed and the path starts with `/', AT_EXECPATH is set
without any translation. If the file is a symbolic link, $ORIGIN may not be
expanded to the actual origin. Use realpath(3) to properly expand $ORIGIN
to its absolute path.

Reviewed by: kib
MFC after: 1 week

Use powerof2(). Remove single-use variable.

Submitted by: Conrad Meyer
Differential Revision: https://reviews.freebsd.org/D1724
MFC after: 1 week

Allow tracing dlfunc() / dlsym() events.

MFC after: 1 week

Do not erronously export 'openat' symbol from rtld.

The symbol leaked after r276630 since lib/libc/sys/openat.c defines
versions for openat using .symver (version script cannot assign two
versions to one symbol), and rtld uses openat. Instead, directly use
__sys_openat().

Reported and tested by: antoine
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Add rtld private interface for dso to detect dynamic loading
vs. static linking.

Tested by: pho, antoine (exp-run)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

The runtime linker needs to include a path to itself in the link map
it exports to the debugger. It currently has two choices: it can use
a compiled-in path (/libexec/ld-elf.so.1) or it can use the path stored
in the interpreter path in the binary being executed. The runtime linker
currently prefers the second. However, this is usually wrong for compat32
binaries since the binary specifies the path of rtld on a 32-bit system
(/libexec/ld-elf.so.1) instead of the actual path (/libexec/ld-elf32.so.1).
For now, always assume the compiled in path (/libexec/ld-elf32.so.1) as
the rtld path and ignore the path in the binary for the 32-bit runtime
linker.

Differential Revision: https://reviews.freebsd.org/D1236
Reviewed by: kib

Revert r274772: it is not valid on MIPS

Reported by: sbruno

Use canonical __PIC__ flag

It is automatically set when -fPIC is passed to the compiler.

Reviewed by: dim, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1179

Always return pathname in dl_iterate_phdr's dlpi_name, as Linux does

Linux LD_ITERATE_PHDR(3):
The dlpi_name field is a null-terminated string giving the
pathname from which the shared object was loaded.

That functionality is much more useful than returning just the short
name.

Approved by: kan
Sponsored by: The FreeBSD Foundation

Optimize r270798, only do the second pass over non-plt relocations
when the first pass found IFUNCs.

Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

IFUNC symbol type shall be processed for non-PLT relocations,
e.g. when a global variable is initialized with a pointer to ifunc.
Add symbol type check and call resolver for STT_GNU_IFUNC symbol types
when processing non-PLT relocations, but only after non-IFUNC
relocations are done. The two-phase proceessing is required since
resolvers may reference other symbols, which must be ready to use when
resolver calls are done.

Restructure reloc_non_plt() on x86 to call find_symdef() and handle
IFUNC in single place.

For non-x86 reloc_non_plt(), check for call for IFUNC relocation and
do nothing, to avoid processing relocs twice.

PR: 193048
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Remove stray newline.

Remove ia64.

This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation

This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h

Discussed at: BSDcan

Fix r264346 for ia64. We need to allocate memory for the function
descriptors in order to relocate RTLD itself. To allocate memory,
we need the pagesizes array initialized, but that happens after
RTLD is relocated. This ordering is important for amd64, but it's
opposite of what ia64 needs. Handle this conflict with the define
called RTLD_INIT_PAGESIZES_EARLY. When defined, obtain the page
sizes before relocating rtld, otherwise do it after.

Add the LD_LIBRARY_PATH_FDS environmental variable.

This variable allows the loading of shared libraries via directory descriptors
rather than via library paths. If LD_LIBRARY_PATH_FDS=3:4:12, the directories
represented by file descriptors 3, 4 and 12 will searched for shared libraries
before the normal path-based mechanisms are used. This allows us to execute
unprivileged binaries from within a Capsicum sandbox even if they require
shared libraries.

Approved by: rwatson (mentor)
Reviewed by: kib
MFC after: 3 weeks
Sponsored by: DARPA/AFRL

- Export the function added in r265456 rather than the non-existent
_rtld_debug_postinit(). [1]
- Use __compiler_membar() instead of inline asm in _r_debug_state() and
_r_debug_postinit(). [2]

Pointy hat to: markj [1]
Reported by: attilio [2]
Discussed with: kib
X-MFC-With: r265456

Add a postinit debugger hook to rtld. This will be used by dtrace(1) to halt
the victim process before its entry point is called, at which point probes
and DOF data are registered with the kernel. The r_debug_state hook cannot
be used for this purpose, as it is called before the program's init routines
are invoked and in particular before DOF data is registered (via drti.o).

Reviewed by: kib
MFC after: 2 weeks

Before calling mmap() on a shared library's text and data sections, rtld
first calls mmap() with the arguments PROT_NONE and MAP_ANON to reserve a
single, contiguous range of virtual addresses for the entire shared library.
Later, rtld calls mmap() with the the shared library's file descriptor
and the argument MAP_FIXED to place the text and data sections within the
reserved range. The rationale for mapping shared libraries in this way is
explained in the commit message for Revision 190885. However, this approach
does have an unintended, negative consequence. Since the first call to
mmap() specifies MAP_ANON and not the shared library's file descriptor, the
kernel has no idea what alignment the vm object backing the file prefers.
As a result, the reserved range's alignment is unlikely to be the same as
the vm object's, and so mapping with superpages becomes impossible. To
address this problem, this revision adds the argument MAP_ALIGNED_SUPER to
the first call to mmap() if the text section is larger than the smallest
superpage size.

To determine if the text section is larger than the smallest superpage
size, rtld must always fetch the page size information. As a result, the
private code for fetching the base page size in rtld's builtin malloc is
redundant. Eliminate it. Requested by: kib

Tested by: zbb (on arm)
Reviewed by: kib (an earlier version)
Discussed with: jhb

Cast Elf_Addr to void * to match the free_aligned() argument type.

Found by: gcc
Sponsored by: The FreeBSD Foundation
MFC after: 6 days

For variant II static TLS, properly align tls segments. Pre-calculate
the max required alignment for the static tls segments, and honor it
when carving the pieces for next module, from the static space. Use
aligned allocator to get properly-aligned dynamic blocks.

Reported by: dt71@gmx.com
Reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Include rtld itself when iterating over loaded ELF objects in
dl_iterate_phdr(3).

Reviewed by: kib

Implement support for the interpose dso flag.

Requested by: bf
Reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (glebius)

Populate .rld_map on MIPS for debuggers

On MIPS the .dynamic section is read-only, so the pointer to rtld
information for debuggers cannot be stored there (in DT_DEBUG).
Instead, a special section .rld_map is used.

Sponsored by: DARPA, AFRL
Approved by: re (delphij)

Make the directory mapping functionality, which was previously only
available in 32-bit compatibility mode, unconditional.

Overhaul the man page, which had evolved more by accretion than by design.

Approved by: re (gjb)
MFC after: 3 weeks

Properly terminate the result string for intermediate results, to
allow the final strcpy() to start at the intended place.

Reported and tested by: pgj
Pointy hat to: kib
MFC after: 3 days

The origin_subst_one() function limits the length of the string to
PATH_MAX after the token substitution. This is wrong, because
origin_subst_one() performs the substitution on the whole rpath and
similar strings, which contain several pathes separated by colon. As
result, long (but correct) rpath consisting of many path elements is
rejected by the function.

Correct the problem by rewriting the origin_subst_one() to perform two
passes, first pass to calculate the number of substitutions to be
performed, and second pass to generate the resulting string. Second
pass allocates the memory for the result based on the count from the
first pass, without enforcing a limit.

Reported and tested by: pgj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

rtld: Fix fd leak with parallel dlopen and fork/exec.

Rtld did not set FD_CLOEXEC on its internal file descriptors; therefore,
such a file descriptor may be passed to a process created by another thread
running in parallel to dlopen() or fdlopen().

No other threads are expected to be running during parsing of the hints
and libmap files but the file descriptors need not be passed to child
processes so add O_CLOEXEC there as well.

This change will break fdlopen() (as used by OpenPAM) on kernels without
F_DUPFD_CLOEXEC (added in July). Note that running new userland on old
kernels is not supported.

Reviewed by: kib

Do not reference z_nodeflib for !objgiven case, thus fixing LD_PRELOAD
for a non-absolute path.

PR: bin/171604
MFC after: 3 days

Do not call process_nodelete with NULL object pointer.

The place where the function is called can be reached if object loading
and relocation fails too, in which case obj pointer will be NULL. Do not
call process_nodelete then, or crash will follow.

Pointy hat to: kan

Pospone the DF_1_NODELETE processing until object DAG is fully loaded.

Trying to up the reference from the load loop risks missing dependencies
that have not been loaded yet.

MFC afer: 1 week
Reported by: nox
Reviewd by: kib

Import the DragonFly BSD commit 4f0bc915b65fcf5a23214f6d221d65c80be68ad4
by John Marino <draco@marino.st>, with the following (edited) commit
message
Date: Sat, 24 Mar 2012 06:40:50 +0100
Subject: [PATCH 1/1] rtld: Implement DT_RUNPATH and -z nodefaultlib

DT_RUNPATH is incorrectly being considered as an alias of DT_RPATH. The
purpose of DT_RUNPATH is to have two different types of rpath: one that
can be overridden by the environment variable LD_LIBRARY_PATH and one that
can't. With the currently implementation, LD_LIBRARY_PATH will always
trump any embedded rpath or runpath tags.

Current path search order by rtld:
==================================
LD_LIBRARY_PATH
DT_RPATH / DT_RUNPATH (always the same)
ldconfig hints file (default: /var/run/ld-elf.so.hints)
/usr/lib

New path search order by rtld:
==============================
DT_RPATH of the calling object if no DT_RUNPATH
DT_RPATH of the main binary if no DT_RUNPATH and binary isn't calling obj
LD_LIBRARY_PATH
DT_RUNPATH
ldconfig hints file
/usr/lib

The new path search matches how the linux runtime loader works. The other
major added feature is support for linker flag "-z nodefaultlib". When
this flag is passed to the linker, rtld will skip all references to the
standard library search path ("/usr/lib" in this case but it could handle
more color delimited paths) except in DT_RPATH and DT_RUNPATH.

New path search order by rtld with -z nodefaultlib flag set:
============================================================
DT_RPATH of the calling object if no DT_RUNPATH
DT_RPATH of the main binary if no DT_RUNPATH and binary isn't calling obj
LD_LIBRARY_PATH
DT_RUNPATH
ldconfig hints file (skips all references to /usr/lib)

FreeBSD notes:
- we fixed some bugs which were submitted to DragonFly and merged there
as commit 1ff8a2bd3eb6e5587174c6a983303ea3a79e0002;
- we added LD_LIBRARY_PATH_RPATH environment variable to switch to
the previous behaviour of considering DT_RPATH a synonym for DT_RUNPATH;
- the FreeBSD default search path is /lib:/usr/lib and not /usr/lib.

Reviewed by: kan
MFC after: 1 month
MFC note: flip the ld_library_path_rpath default value for stable/9

Ensure that for the object which is a dependency for some filtee,
relocations are performed before the object's initializer is called.
When dlopen()ing an object, relocate the whole DAG rooted in the
object instead of only relocating the object itself and list of newly
loaded dependencies.

Reversed sequence currently can occur if the same object is a
dependency for both filtee and filter, since filtees are loaded
typically during the relocation processing, when some filter
dependencies might be already loaded but not relocated yet.

Reported and tested by: swills
Reviewed by: kan
MFC after: 1 week

Work around a situation where symlook_obj() could be called for the
object for which digest_dynamic1() was not done yet. Just return
EINVAL and do not try to dereference NULL buckets hash array.

This seems to happen on ia64 for rtld object itself, where the
R_IA_64_FPTR64LSB relocations require symbol lookup. The dynamic
linker itself does not rely on identity of the C-level function
pointers (i.e. function descriptors).

Reported and reviewed by: marcel
MFC after: 8 days

Add GNU hash support for rtld.

Based on dragonflybsd support for GNU hash by John Marino <draco marino st>
Reviewed by: kan
Tested by: bapt
MFC after: 2 weeks

Split the symlook_obj1 into a loop iterating over the ELF object symbol
hash elements, and a helper matched_symbol() which match the given hash
entry and request, performing needed type and version checks.

Based on dragonflybsd support for GNU hash by John Marino <draco marino st>
Reviewed by: kan
Tested by: bapt
MFC after: 2 weeks

Propagate the current state of rtld_bind_lock to dlopen_object() calls
through the filter loading call chain. This fixes attempts to
write-lock the already locked rtld_bind_lock when filter loading is
initiated by relocation of dlopening dso.

Reported and tested by: Taku YAMAMOTO <taku tackymt homeip net>
MFC after: 1 week

Properly handle absent AT_CANARY aux entry.

Submitted by: Andrey Zonov <andrey zonov org>
MFC after: 3 days

Remove extra blank line from revious commit.

Submitted by: trema

Do not try to adjust stacks if dlopen_object is called too early.

This is a follow-up to r233231, which fixed similar issue with
object initialization code.

Reviewed by: kib
MFC after: 1 week (with 233231)

Fix ia64 build after r233655.

MFC after: 1 week

Import DragonFly BSD commit

From: Sascha Wildner <saw@online.de>
Date: Fri, 2 Mar 2012 09:15:56 +0000 (+0100)
Subject: rtld: Add a special case in do_dlsym() for TLS stored symbols.
X-Git-Url: http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff_plain/1388aaafe730c85693936aaf9bc6d83fc2d981be?hp=bca4412595a8979ab9f1bf36068c806ce88a667a

rtld: Add a special case in do_dlsym() for TLS stored symbols.

Submitted-by: Markus Pfeiffer <markus.pfeiffer@morphism.de>

Discussed with: kan
MFC after: 1 week

Prevent rtld_verify_object_versions() from being called several times
for the same object. This can happen when object is a dependency of the
dlopen()ed dso. When called several times, we waste time due to unneeded
processing, and memory, because obj->vertab is allocated anew on each
iteration.

Reviewed by: kan
MFC after: 2 weeks

Remove libssp_nonshared from the rtld linking set. The only use for the
library was definition for the weak alias of __stack_chk_fail.

No objections from: kan
MFC after: 2 weeks

Provide short-circuit exit(3) implementation for rtld.
There is no atexit finalizers in rtld to call on rtld exiting (due to errors).

Submitted by: bde
No objections from: kan
MFC after: 2 weeks

Use xmalloc() instead of malloc() in the places where malloc() calls
are assumed to not fail.

Make the xcalloc() calling conventions follow the calloc(3) calling
conventions and replace unchecked calls to calloc() with calls to
xcalloc().

Remove redundand declarations from xmalloc.c, which are already
present in rtld.h.

Reviewed by: kan
Discussed with: bde
MFC after: 2 weeks

Fix several problems with our ELF filters implementation.

Do not relocate twice an object which happens to be needed by loaded
binary (or dso) and some filtee opened due to symbol resolution when
relocating need objects. Record the state of the relocation
processing in Obj_Entry and short-circuit relocate_objects() if
current object already processed.

Do not call constructors for filtees loaded during the early
relocation processing before image is initialized enough to run
user-provided code. Filtees are loaded using dlopen_object(), which
normally performs relocation and initialization. If filtee is
lazy-loaded during the relocation of dso needed by the main object,
dlopen_object() runs too earlier, when most runtime services are not
yet ready.

Postpone the constructors call to the time when main binary and
depended libraries constructors are run, passing the new flag
RTLD_LO_EARLY to dlopen_object(). Symbol lookups callers inform
symlook_* functions about early stage of initialization with
SYMLOOK_EARLY. Pass flags through all functions participating in
object relocation.

Use the opportunity and fix flags argument to find_symdef() in
arch-specific reloc.c to use proper name SYMLOOK_IN_PLT instead of
true, which happen to have the same numeric value.

Reported and tested by: theraven
Reviewed by: kan
MFC after: 2 weeks

Rtld on diet 3.

Stop using strerror(3) in rtld, which brings in msgcat and stdio.
Directly access sys_errlist array of errno messages with private
rtld_strerror() function.

Now,
$ size /libexec/ld-elf.so.1
text data bss dec hex filename
96983 2480 8744 108207 1a6af /libexec/ld-elf.so.1

Reviewed by: dim, kan
MFC after: 2 weeks

Rtld on diet part 1:

Provide rtld-private implementations of __stack_chk_guard,
__stack_chk_fail() and __chk_fail() symbols, to be used by functions
linked from libc_pic.a. This avoids use of libc stack_protector.c,
which pulls in syslog(3) and stdio as dependency.

Also, do initialize rtld-private copy __stack_chk_guard, previously
libc-provided one was not initialized, since we do not call rtld
object _init() methods.

Reviewed by: kan
MFC after: 3 weeks

Amend r232857, now dropping the casts entirely, as they were not
necessary at all.

Submitted by: stefanf

Fix the following warning/error with clang:

libexec/rtld-elf/rtld.c:1898:22: error: comparison between pointer and integer ('Elf_Addr *' (aka 'unsigned int *') and 'Elf_Addr' (aka 'unsigned int')) [-Werror]
if (preinit_addr == (Elf_Addr)NULL)
~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~
libexec/rtld-elf/rtld.c:2039:16: error: comparison between pointer and integer ('Elf_Addr *' (aka 'unsigned int *') and 'Elf_Addr' (aka 'unsigned int')) [-Werror]
if (init_addr != (Elf_Addr)NULL) {
~~~~~~~~~ ^ ~~~~~~~~~~~~~~

Reviewed by: kib

Add support for preinit, init and fini arrays. Some ABIs, in
particular on ARM, do require working init arrays.

Traditional FreeBSD crt1 calls _init and _fini of the binary, instead
of allowing runtime linker to arrange the calls. This was probably
done to have the same crt code serve both statically and dynamically
linked binaries. Since ABI mandates that first is called preinit
array functions, then init, and then init array functions, the init
have to be called from rtld now.

To provide binary compatibility to old FreeBSD crt1, which calls _init
itself, rtld only calls intializers and finalizers for main binary if
binary has a note indicating that new crt was used for linking. Add
parsing of ELF notes to rtld, and cache p_osrel value since we parsed
it anyway.

The patch is inspired by init_array support for DragonflyBSD, written
by John Marino.

Reviewed by: kan
Tested by: andrew (arm, previous version), flo (sparc64, previous version)
MFC after: 3 weeks

Optimize tls_get_addr_common(). The change provides around 30% speedup
for TLS microbenchmark using global-dynamic TLS model on amd64 (which is
default for PIC dso objects).

Split the slow path into tls_get_addr_slow(), for which inlining is
disabled. This prevents the registers spill on tls_get_addr_common()
entry.

Provide static branch hint to the compiler, indicating that slow path
is not likely to be taken.

While there, do some minimal style adjustments.

Reported and tested by: davidxu
MFC after: 1 week

Add thread-local storage support for ARM to rtld-elf

Reviewed by: cognet
Obtained from: NetBSD

Add missed EOL when die() was converted to use rtld_fdputstr() instead
of errx().

Reported by: amdmi3
PR: bin/165075
MFC after: 3 days

Switch MIPS TLS implementation to Variant I

Add support for GNU RELRO.

Submitted by: John Marino <draco marino st>
MFC after: 2 weeks

Implement fdlopen(3), an rtld interface to load shared object by file
descriptor.

Requested and tested by: des (previous version)
Reviewed by: des, kan (previous version)
MFC after: 2 weeks

Postpone the resolution of IRELATIVE relocations and IFUNC-targeted
relocations until tls is initialized and stacks permissions correctly
set. This allows the ifunc to call malloc(3) and some other heavy
services.

Add debug banner.

MFC after: 3 days

_rtld_bind() read-locks the bind lock, and possible plt resolution
from the dispatcher would also acquire bind lock in read mode, which
is the supported operation. plt is explicitely designed to allow safe
multithreaded updates, so the shared lock do not cause problems.

The error in r228435 is that it allows read lock acquisition after the
write lock for the bind block. If we dlopened the shared object that
contains IRELATIVE or jump slot which target is STT_GNU_IFUNC, then
possible recursive plt resolve from the dispatcher would cause it.

Postpone the resolution for irelative/ifunc right before initializers
are called, and drop bind lock around calls to dispatcher. Use
initlist to iterate over the objects instead of the ->next, due to
drop of the bind lock in iteration.

For i386/reloc.c:reloc_iresolve(), fix calculation of the dispatch
function address for dso, by taking into account possible non-zero
relocbase.

MFC after: 3 weeks

Add support for STT_GNU_IFUNC and R_MACHINE_IRELATIVE GNU extensions to
rtld on 386 and amd64. This adds runtime bits neccessary for the use
of the dispatch functions from the dynamically-linked executables and
shared libraries.

To allow use of external references from the dispatch function, resolution
of the R_MACHINE_IRESOLVE relocations in PLT is postponed until GOT entries
for PLT are prepared, and normal resolution of the GOT entries is finished.
Similar to how it is done by GNU, IRELATIVE relocations are resolved in
advance, instead of normal lazy handling for PLT.

Move the init_pltgot() call before the relocations for the object are
processed.

MFC after: 3 weeks

Fix fd leak.

Reported by: slonoman2011 yandex ru
MFC after: 1 week

Setting up TLS block for the main thread must be done after the
relocations are processed, since tls initialization section might be
itself subject for relocations. Only set up of the block is postponed,
the tls block offsets are allocated before relocation processing, since
TLS-related relocations may need offsets ready.

Reported by: ale
PR: threads/161344
Reviewed by: kan
MFC after: 1 week

Use the proper dynamic tls block to calculate the tls variable address
in case tls data generation was updated.

PR: misc/160721
Submitted by: "Thinker K.F. Li" <thinker codemud net>
Tested by: flo
Approved by: re (bz)
MFC after: 1 week

When libexec/rtld-elf/rtld.c is compiled with clang, the r_debug_state()
function (a hook necessary for gdb support), is inlined, but since the
function contains no code, no calls to it are generated. When gdb is
debugging a dynamically linked program, this causes backtraces to be
corrupted.

Fix it by marking the function __noinline, and inserting an empty asm
statement, that pretends to clobber memory. This forces the compiler to
emit calls to r_debug_state() throughout rtld.c.

Approved by: re (kib)

Rtld links with the specially built pic static libc library to get some
C runtime services, like printf(). Unfortunately, the multithread-safeness
measures in the libc do not work in rtld environment.

Rip the kernel printf() implementation and use it in the rtld instead of
libc version. This printf does not require any shared global data and thus
is mt-safe. Systematically use rtld_printf() and related functions, remove
the calls to err(3).

Note that stdio is still pulled from libc due to libmap implementaion using
fopen(). This is safe but unoptimal, and can be changed later.

Reported and tested by: pgj
Diagnosed and reviewed by: kan (previous version)
Approved by: re (bz)

Fix clang warnings.

Approved by: philip (mentor)

rtld: eliminate double call to close(2) that may occur in load_object

The second close(2) call resulted in heisenbugs in some multi-threaded
applications where e.g. dlopen(3) call in one thread could close a file
descriptor for a file having been opened in other thread concurrently.

My litmus test for this issue was an openoffice.org build.

Reviewed by: jhb
MFC after: 2 weeks

Use sigsetjmp/siglongjmp with disabled signal mask access for
lock upgrade in rtld. There is no need to care about the mask,
which causes a lot of unneeded sigprocmask(2) calls during each
symbol lookup.

Fix grammar in comment.

Pointy hat to: kib
MFC after: 1 week

Make ldd(1) work when versioned dependency file is cannot be loaded.
Instead of aborting in locate_dependency(), propagate the error to
caller. The rtld startup function does the right thing with an error
from rtld_verify_versions(), depending on the mode of operation.

Reported by: maho
In collaboration with: kan
MFC after: 1 week

Eliminate the use of symlook_needed function in favor of DAGS.

Place elements on DAG lists in breadth-first order. This allows us to
walk pre-built list in all cases where breadth-first dependency chain
enumeration is required.

Fix dlsym on special handle obtained by dlopen(NULL, ...) to do what
comment claims it does. Take advantage of recently added symlook_global
function to iterate over main objects and global DAGs lists properly in
search of a symbol. Since rtld itself provides part of the global
namespace, search rtld_obj too.

Remove recursion from init_dag and symlook_needed functions. Use
symlook_needed for ELF filtee processing only and change lookup order
used in the function to match the order used by Solaris runtime linker
under same circumstances. While there, fix weak symbol handling in the
loop so that we return the first weak symbol definition if no strong one
was found, instead of the last one.

Reviewed by: kib
MFC after: 1 month

Remove SuperH architecture from a comment as we do not support it.
Presumably it was leaked from NetBSD together with rtld-elf mips support.

Approved by: kib (mentor)
MFC after: 3 days

When loading dso without PT_GNU_STACK phdr, only call
__pthread_map_stacks_exec() on architectures that allow executable
stacks.

Reported and tested by: marcel (ia64)

Fix get_program_var_addr() when type of the resolved symbol is functional.
Use make_function_pointer then, otherwise ia64 is broken.

Reported and tested by: marcel

Add my copyright.

Discussed with: kan

get_program_var_addr() must prefer the strong symbol to the weak
one. Search global objects, together with main object and
dependencies, for the requested symbol.

Move the common code from symlook_default() into new helper
symlook_global(), and use it both in symlook_global() and
get_program_var_addr().

Supply lock state to get_program_var_addr().

Reviewed by: kan
Tested by: Mykola Dzham <i levsha me>

In rtld, read the initial stack access mode from AT_STACKPROT as set
by kernel, and parse PT_GNU_STACK phdr from linked and loaded dsos.

If the loaded dso requires executable stack, as specified by PF_X bit
of p_flags of PT_GNU_STACK phdr, but current stack protection does not
permit execution, the __pthread_map_stacks_exec symbol is looked up
and called. It should be implemented in libc or threading library and
change the protection mode of all thread stacks to be executable.

Provide a private interface _rtld_get_stack_prot() to export the stack
access mode as calculated by rtld.

Reviewed by: kan

Fix an apparent cop-and-paste mistake in previous commit.

This makes dlsym(RTLD_DEFAULT) work properly again.

Implement support for ELF filters in rtld. Both normal and auxillary
filters are implemented.

Filtees are loaded on demand, unless LD_LOADFLTR environment variable
is set or -z loadfltr was specified during the linking. This forces
rtld to upgrade read-locked rtld_bind_lock to write lock when it
encounters an object with filter during symbol lookup.

Consolidate common arguments of the symbol lookup functions in the
SymLook structure. Track the state of the rtld locks in the
RtldLockState structure. Pass local RtldLockState through the rtld
symbol lookup calls to allow lock upgrades.

Reviewed by: kan
Tested by: Mykola Dzham <i levsha me>, nwhitehorn (powerpc)

If dlclose() is called recursively from a _fini() function, the inner
dlclose() call may unload the object of the outer call prematurely
because objects are unreferenced before _fini() calls.

Fix this by unreferencing objects after calling objlist_call_fini() in
dlclose(). Therefore objlist_call_fini() now calls the fini function if
the reference count of an object is 1. In addition we must restart the
list_fini traversal after every _fini() call because another dlclose()
call might have modified the reference counts.

Add an XXX comment to objlist_call_fini() about possible race with
dlopen().

PR: 133246, 149464
Reviewed by: kan, kib

In r214728, if dlopen() is called for the object that has been already
loaded as a dependency and marked -z nodlopen, object' DAG is already
initialized by load_needed_objects(). Due to this, the init_dag() call
from dlopen() does not increment refcount for the object [1].

Change init_dag() to not increment DAG refcount. Require explicit calls
to ref_dag() to increment, and assert that ref_dag() and unref_dag()
are called for root that has dag initialized. To fix the noted issue,
unconditionally call both init_dag() and ref_dag() in dlopen() for the
case when the object was already loaded, making it similar to the case
of newly loaded object.

Noted by: jh [1]
Reviewed by: jh, kan
MFC after: 6 days

Fix style.

MFC after: 6 days

If dlopen() is called for the dso that has been already loaded as a
dependency, then the dso never has its DAG initialized. Empty DAG
makes ref_dag() call in dlopen() a nop, and the dso refcount is off
by one.

Initialize the DAG on the first dlopen() call, using a boolean flag
to prevent double initialization.

From the PR (edited):
Assume we have a library liba.so, containing a function a(), and a
library libb.so, containing function b(). liba.so needs functionality
from libb.so, so liba.so links in libb.so.

An application doesn't know about the relation between these libraries,
but needs to call a() and b(). It dlopen()s liba.so and obtains a
pointer to a(), then it dlopen()s libb.so and obtains a pointer to b().

As soon as the application doesn't need a() anymore, it dlclose()s liba.so.

Expected result: the pointer to b() is still valid and can be called
Actual result: the pointer to b() has become invalid, even though the
application did not dlclose() the handle to libb.so. On calling b(), the
application crashes with a segmentation fault.

PR: misc/151861
Based on patch by: jh
Reviewed by: kan
Tested by: Arjan van Leeuwen <freebsd-maintainer opera com>
MFC after: 1 week

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Check for undefined weak symbols during PLT binding on powerpc64, and do
not attempt to copy NULL function descriptors. This fixes LD_BIND_NOW on
powerpc64 after r211706.

Reviewed by: kib

Remove exports table. Export control by the version script is enough.

Reviewed by: kan
MFC after: 3 weeks

On shared object unload, in __cxa_finalize, call and clear all installed
atexit and __cxa_atexit handlers that are either installed by unloaded
dso, or points to the functions provided by the dso.

Use _rtld_addr_phdr to locate segment information from the address of
private variable belonging to the dso, supplied by crtstuff.c. Provide
utility function __elf_phdr_match_addr to do the match of address against
dso executable segment.

Call back into libthr from __cxa_finalize using weak
__pthread_cxa_finalize symbol to remove any atfork handler which
function points into unloaded object.

The rtld needs private __pthread_cxa_finalize symbol to not require
resolution of the weak undefined symbol at initialization time. This
cannot work, since rtld is relocated before sym_zero is set up.

Idea by: kan
Reviewed by: kan (previous version)
MFC after: 3 weeks

Introduce implementation-private rtld interface _rtld_addr_phdr, which
fills struct dl_phdr_info for the shared object that contains the
specified address, if any.

Idea and reviewed by: kan
MFC after: 3 weeks

Provide the starting image with the pointer to ELF aux vector. It is
written into the __elf_aux_vector variable, if the symbol is present.

Idea from: kan
Tested by: marius (sparc64)
MFC after: 1 month

Use the newly provided aux vectors to get pagesize and osreldate information.
Use local version of getpagesize(), rtld_getpagesize() in private allocator.
Override the __getosreldate() previously fetched from libc_pic.a with
local version that uses aux value if present. Note that __getosreldate()
is used by rtld indirectly, by mmap(2) libc wrapper.

To be able to utilize aux, split digest_dynamic() for use by init_rtld()
into two parts, where the first one does not call malloc(), and the
second part uses it. init_rtld() is able to initialize global variables
before digest_dynamic2() calls. In particular, pagesize and osreldate are
set up from the aux values.

Now, rtld avoids (two) sysctl calls in startup.

Tested by: marius (sparc64)
MFC after: 1 month

Revert r209544. It papers over a binutils bug, and is not even a reliable
fix for it.

The bug occurs when using the --as-needed flag to ld in the presence of
synthetic linker-generated symbols that reference symbols defined in
linked-to shared libraries with versioned symbols. When the only symbols
used from a library fall into this category, ld will drop the DT_NEEDED
entry for it, but retain the versioning information. This bug is best
fixed/hacked around in binutils, not in rtld.

Discussed with: kan

Ignore versioned dependencies on shared objects to which we do not link.
This fixes an error with files like this created by GNU ld under certain
circumstances.

Plug possible memory leak.

Found by: Coverity
MFC after: 2 weeks

Only use the cache after the early stage of loading. This is
because calling mmap() etc. may use GOT which is not set up
yet. Use calloc() instead of mmap() in cases where this
was the case before (sparc64, powerpc, arm).

Submitted by: Dimitry Andric (dimitry andric com)
Reviewed by: kan
Approved by: ed (mentor)

MFC r205606
Remove const'ness from dlerror(3) prototype, for consistency with POSIX.

Approved by: cognet

- Remove const'ness from dlerror(3) prototype, for consistency with POSIX.

Approved by: cognet
MFC after: 1 week

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Improve TLS variant I:
o Use obj->tlsinitsize to determine whether there's initialized data.
o If obj->tlssize > obj->tlsinitsize, then bzero uninitialized data.
o Don't exclude variant I from the work-around in free_tls_offset().

MFC r199829:
Implement rtld part of the support for -z nodlopen (see ld(1)).

MFC r199877:
Allow to load not-openable dso when tracing. This fixes ldd on such dso or
dso linked to non-openable object.
Remove '\n' at the end of error message.
End comments with dot.

Merge r199270 from head to stable/8:

Fix white space in rtld runtime error printf.

MFC r199979: Fix local root vulnerability.

Fix local root vulnerability.

Security: Advisory will be coming soon.
X-MFC-After: 30 seconds

MFC r199828:
Flag controlling origin expansion in DT_FLAGS is DF_ORIGIN, not DF_1_ORIGIN.

Allow to load not-openable dso when tracing. This fixes ldd on such dso or
dso linked to non-openable object.
Remove '\n' at the end of error message.
End comments with dot.

MFC after: 3 weeks (together with r199829)

Implement rtld part of the support for -z nodlopen (see ld(1)).

Reviewed by: kan
MFC after: 3 weeks

Flag controlling origin expansion in DT_FLAGS is DF_ORIGIN, not DF_1_ORIGIN.

Reviewed by: kan
MFC after: 3 days

Fix white space in rtld runtime error printf.

MFC after: 3 days

MFC r197931:
Apply relocations for PIE binary ELF data structures pointers in rtld.

Approved by: re (kensmith)

Calculate relocation base for the main object, and apply the relocation
adjustment for all virtual addresses encoded into the ELF structures of
it. PIE binary could and should be loaded at non-zero mapbase.

For sym_zero pseudosymbol used as a return value from find_symdef()
for undefined weak symbols, st_value also should be adjusted, since
_rtld_bind corrects symbol values by relocbase.

Discussed with: bz
Reviewed by: kan
Tested by: bz (i386, amd64), bsam (linux)
MFC after: some time

Implement RTLD_NOLOAD flag for dlopen(3).

Requested and tested by: jkim
Reviewed by: kan
Approved by: re (kensmith)

Fix a typo in the same comment, one line below.

Submitted by: bf1783 googlemail com

Fix typo in comment.

Submitted by: Christoph Mallon

Allow order of initialization of loaded shared objects to be
altered through their .init code. This might happen if init
vector calls dlopen on its own and that dlopen causes some not
yet initialized object to be initialized earlier as part of that
dlopened DAG.

Do not reset module reference counts to zero on final fini vector
run when process is exiting. Just add an additional parameter to
force fini vector invocation regardless of current reference count
value if object was not destructed yet. This allows dlclose called
from fini vector to proceed normally instead of failing with handle
validation error.

Reviewed by: kib
Reported by: venki kaps

FreeBSD returns main object handle from dlopen(NULL, ...) calls.
dlsym seaches using this handle are expected to look for symbol
definitions in all objects loaded at the program start time along
with all objects currently in RTLD_GLOBAL scope.

Discussed with: kib
Reported by: Maho NAKATA
MFC after: 2 weeks

Allow the NULL, RTLD_SELF and RTLD_NEXT handles to work with dlfunc(3).
dlfunc() called dlsym() to do the work, and dlsym() determines the dso
that originating the call by the return address. Due to this, dlfunc()
operated as if the caller is always the libc.

To fix this, move the dlfunc() to rtld, where it can call the internal
implementation of dlsym, and still correctly fetch return address.
Provide usual weak stub for the symbol from libc for static binaries.
dlfunc is put to FBSD_1.0 symver namespace in the ld.so export to
override dlfunc@FBSD_1.0 weak symbol, exported by libc.

Reported, analyzed and tested by: Tijl Coosemans <tijl ulyssis org>
PR: standards/133339
Reviewed by: kan