ld.so should stop calling msyscall(2), since it is fully neutered and
about to be removed. Please be careful building through this, you need
a kernel at least March 29th or so to build through it, otherwise use
snapshots to cross over.
ok various people

very ugly whitespaces

Read PT_OPENBSD_SYSCALLS in libc.so, and convert it to a table for
pinsyscalls(2).
ok kettenis

whitespaces

Skip the _dl_msyscall() invocation if tracing library loading.

Problem noted by gnezdo@
ok millert@

validate alignment of ELF program headers

Accumulate intermediate imutables locally before applying

OK deraadt

The next step for mimmutable(). ld.so figures out what regions of memory
of startup shared library mappings can be made immutable, and also does
this for dlope() RTLD_NODELETE and subsidiary libraries. Complexity in this
diff is due to the GNU_RELRO and OPENBSD_MUTABLE sections.
Tested in snaps for about 3 weeks, with some bootstrap related pain felt in ports
ok kettenis, much help from others.

dtors were broken by trying to reuse DF_1_NODELETE to hint that this
library would never unload, and could be immutable. Pass a seperate
flag for our purposes
Noticed from regress tests by anton, ok kettenis

Support RTLD_NOLOAD in ld.so. From guenther@. OK jca@ guenther@

Prep .c files for removing the #includes from */archdep.h
* replace #include "archdep.h" with #includes of what is used, pulling in
"syscall.h", "util.h", and "archdep.h" as needed
* delete #include <sys/syscall.h> from syscall.h
* only pull in <sys/stat.h> to the three files that use _dl_fstat(),
forward declare struct stat in syscall.h for the others
* NBBY is for <sys/select.h> macros; just use '8' in dl_printf.c
* <machine/vmparam.h> is only needed on i386; conditionalize it
* stop using __LDPGSZ: use _MAX_PAGE_SHIFT (already used by malloc.c)
where necessary
* delete other bogus #includes, order legit per style: <sys/*> then
<*/*>, then <*>, then "*"

dir.c improvement from jsg@
ok and testing assistance deraadt@

On i386 don't attempt to map shared libraries in low memory when
a large executable's .text section crosses the 512MB exec line.

Executables that have MAXTSIZ > 64MB can map above the default
512MB exec line. When this happens, shared libs that attempt to map
into low memory will find their .data section can not be mapped. ld.so
will attempt to remap the share lib at higher addresses until it can be
mapped. For very large executables like chrome this process is very
time consuming. This change detects how much of the executable's
.text section exceeds 512MB and uses that as the initial hint for
shared libs to map into which avoids attempting to map into blocked
memory.

okay deraadt@

When loading a library, mmap(2) may fail. Then everything gets
unmapped and ld.so tries again with different random address layout.
In this case, use the new libc executable address for msyscall(2),
not one from the first try. Fixes sporadic bogus syscall on i386.
OK deraadt@

print addresses upon msyscall failure, for now

Sigh, fix i386 msyscall() case to permission the correct address range.

Repurpose the "syscalls must be on a writeable page" mechanism to
enforce a new policy: system calls must be in pre-registered regions.
We have discussed more strict checks than this, but none satisfy the
cost/benefit based upon our understanding of attack methods, anyways
let's see what the next iteration looks like.

This is intended to harden (translation: attackers must put extra
effort into attacking) against a mixture of W^X failures and JIT bugs
which allow syscall misinterpretation, especially in environments with
polymorphic-instruction/variable-sized instructions. It fits in a bit
with libc/libcrypto/ld.so random relink on boot and no-restart-at-crash
behaviour, particularily for remote problems. Less effective once on-host
since someone the libraries can be read.

For static-executables the kernel registers the main program's
PIE-mapped exec section valid, as well as the randomly-placed sigtramp
page. For dynamic executables ELF ld.so's exec segment is also
labelled valid; ld.so then has enough information to register libc's
exec section as valid via call-once msyscall(2)

For dynamic binaries, we continue to to permit the main program exec
segment because "go" (and potentially a few other applications) have
embedded system calls in the main program. Hopefully at least go gets
fixed soon.

We declare the concept of embedded syscalls a bad idea for numerous
reasons, as we notice the ecosystem has many of
static-syscall-in-base-binary which are dynamically linked against
libraries which in turn use libc, which contains another set of
syscall stubs. We've been concerned about adding even one additional
syscall entry point... but go's approach tends to double the entry-point
attack surface.

This was started at a nano-hackathon in Bob Beck's basement 2 weeks
ago during a long discussion with mortimer trying to hide from the SSL
scream-conversations, and finished in more comfortable circumstances
next to a wood-stove at Elk Lakes cabin with UVM scream-conversations.

ok guenther kettenis mortimer, lots of feedback from others
conversations about go with jsing tb sthen

Convert the child_list member from a linked list to a vector.

ok mpi@

On i386, ensure that the first PT_LOAD segment is below the W^X
line unless it is writable.

lld places read-only sections below the gap so this is needed
to be able to retain W^X with lld. Note however the read-only
sections below the W^X line are now executable on pre-NX
machines and a possible source of gadgets. This is a change
from Gnu ld where RO sections were ordered above the W^X line
and not executable.

okay drahn@ kettenis@ deraadt@

Everyone knows this as ld.so, nor by the ancient name rtld.
ok guenther

On fatal errors, kill ourselves with thrkill(0,9,NULL) instead of
simply exiting, via helper functions _dl_die(), _dl_diedie(), and
_dl_oom().

prompted by a complaint from jsing@
ok jsing@ deraadt@

the slimmed down random functions inside ld.so are strict clones of the
libc arc4random API, so call them _dl_{arc4random,arcrandombuf}
ok tedu guenther

Look for a PT_GNU_RELRO section per object and, if present, mprotect that
range instead of the [__got_start, __got_end) range.
On many archs this will cover _DYNAMIC too, so move up the DT_DEBUG handling
to before relocations and the mprotect are done.

ok kettenis@

Remove prebind support: binding to symbol table indices is too fragile
for our development process.

ok kettenis@ deraadt@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Export environ and __progname, making the latter a copy of just the filename
portion like crt0 does. This is prep for eliminating _dl_fixup_user_env()
Mark almost everything in resolve.h as hidden, to improve code generation.

ok kettenis@ mpi@ "good time" deraadt@

assign pointers to NULL rather than 0

Fix unloading of load groups when the last reference wasn't on the
load_object but rather some descendent. Detect that case in
_dl_unload_shlib() and switch to unloading the entire group.

Based on partial analyses by Henri Kemppainen (duclare (at) guu.fi)
and Peter Hajdu (peter.ferenc.hajdu (at) gmail.com)
ok millert@

delete annoying white space that annoy

<sys/param.h> to <limits.h> conversion. Verified binaries
ok millert, thanks to doug for process advice

check all memory allocations; ok miod@ guenther@

Use O_CLOEXEC to make sure fork+exec in a threaded process can't
see the fds used by dlopen()

ok otto@ miod@

Move to a non-zeroing _dl_malloc, a _dl_calloc and _dl_reallocarry and
fix _dl_strdup to return NULL instead of crash; ok deraadt@

Add support for .openbsd.randomdata sections and PT_OPENBSD_RANDOMIZE
segments to the kernel, ld (2.15), and ld.so. Tested on alpha, amd64,
i386, macppc, and sparc64 (thanks naddy, mpi, and okan!).

Idea discussed for some time; committing now for further testing.
ok deraadt

Add a new mmap(2) flag __MAP_NOREMAP for use with MAP_FIXED to
indicate that the kernel should fail with MAP_FAILED if the specified
address is not currently available instead of unmapping it.

Change ld.so on i386 to make use of __MAP_NOREMAP to improve
reliability.

__MAP_NOREMAP diff by guenther based on an earlier diff by Ariane;
ld.so bits by guenther and me
bulk build stress testing of earlier diffs by sthen
ok deraadt; committing now for further testing

Fix loaded object sod matching: when we load libfoo.so.X.Y into
memory, we should be able to match other requests for libfoo.so.X.Z
against that same object.

ok kurt, kettenis

Refuse to load ELF objects that contain a PT_TLS program header.
Otherwise the binary assumes that the requested TLS storage has been
allocated and will happily use it, resulting in unwanted memory corruption.

ok guenther@

Don't mmap 0 byte areas, treat them as a noop instead.

ok miod@

Add support for getting some flags from DT_FLAGS_1: new flags
DF_1_NODELETE and DF_1_INITFIRST, as well as DF_1_NOW and DF_1_GLOBAL.

Committing for kurt@ who worked out the final version; ok guenther@ drahn@

Fix error message when ld.so ends up loading a different than expected
library, but other library needs the one loaded. mostly ok kurt@

Search loaded libs first and add support for SONAME matching. ok drahn@

Fix mmap() error checking to be correct 64-bit addresses. Consistently
use _dl_mmap_error() to check for mmap() errors. Adjust datatypes of
some local vars for 64-bit safety.

okay millert@ drahn@

Constantly fill in the program header pointer and count in elf_object_t
for all objects which simplifies phdr usage in a few places.
"go for it" drahn@

pretty this up; ok drahn

prebind - how to prelink a binary without throwing security out the window

Prelink fixes the address of libraries making 'return to libc' attacks trival,
prebind uses a different method to achieve most of the same gains, however
without adding any security conerns.

Still under development, now in-tree.

add RTLD_NOW support to dlopen and propogate -z now to dep libs.
ok drahn@

Split grpsym_list creation away from child_list creation and change
grpsym_list order to match Sun's docs. Also corrects bugs where
grpsym_list was either not created or partially created.

introduce object ref count macros (suggested by dale). no functional
change.

separate load group references from dep lib child/dload lists. move load
group refs to own per object ref counter (grprefcount) and list
(grpref_list). corrects more complex load group ref cases and side effects
from initial implementation. design ideas and ok drahn@

refcount corrections: count common dep libs once and centralize dep lib
refcount increments to _dl_link_sub. adjust _dl_notify_unload_shlib to
match new refcount method. ok drahn@

handle references to load groups caused by dlopen()ing of depenant
members of the load group. work by kurt@ and myself

correct last commit, in both files.

increment refcount when opening a lib that is already open (dev/inode
case) ok drahn@

Fully unload dependant libraries, fixes gphoto2 bug.

Rework symbol lookup to more closely match sun's documentation, now
treats dlopens as load groups. ok kurt@

removed unused var, pointed out by wilfried@

unload is not to be recursive anymore, previous diff change method but
this code was missed, fixes issue seen by tedu and millert hopefully php too.

fixes for dlclose, ok kettenis@

Recommit the destructor order fix, now that the amd64 bug was fixed.
'no problem' pval@

backout -- breaks at least amd64; spotted by marc

Do a better job of running destructors in the right order.

Code reorganization, move copied code in library.c and library_mquery.c
into its own file. no functional change.

fix PR3944 on i386 (ugh), from library.c
--
Fix some problems related to LD_LIBRARY_PATH parsing where it would not
correctly deal with current directory searches specified by "::", ":foo" or
"foo:"
--
ok mickey@

Fix an issue where a shared library could be loaded at two different
locations by resolving all dlopens back to a dev/inode.

i.e. Don't load a library if the dev/inode it stats back to matches one
already in our list.

fix started (and ok'ed) drahn@. ok deraadt@.
"doesn't break anything yet" pval@ art@ brad@

Fix PR 3371, symbol lookup in dlopen()ed objects is not correct. Correct
behavior for RTLD_GLOBAL/RTLD_LOCAL is now supported. ok espie@

Fix print if minor of library used is is less than requested. ok tdeval@

various proto, ansi, and knf repair. tested on all architectures that
use it. (build may require make cleandir because of .depend balony)

Correct library search algorithm, wrt versioned objects

Dynamic linking random order fixes. This enables random library ordering.
Tested by naddy@ and others.

pefo 3/4 licence cleanups

When loading a shared object or libraries dependant object, load them
in random order. This will reduce the possiblity of a buffer overflow
being able to predict the addresss of useful code. Can be disabled
with the LD_NORANDOM environment variable for debugging purposes.
ok deraadt.

Change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built, booted, and 'make includes' before building
ld.so with this change.

backout mquery change, something broke when not combined with a different diff.

change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built and installed before building ld.so with this change.
ok millert@ tedu@

Fix the load_size which is used by ldd to print end address.

Adjusting the start is wrong. The value returned by mquery() should be used
here. Fixes a bug with emacs21. tested by marc@ and others.

mquery support, currently for i386 only.

very ugly whitespaces

Read PT_OPENBSD_SYSCALLS in libc.so, and convert it to a table for
pinsyscalls(2).
ok kettenis

whitespaces

Skip the _dl_msyscall() invocation if tracing library loading.

Problem noted by gnezdo@
ok millert@

validate alignment of ELF program headers

Accumulate intermediate imutables locally before applying

OK deraadt

The next step for mimmutable(). ld.so figures out what regions of memory
of startup shared library mappings can be made immutable, and also does
this for dlope() RTLD_NODELETE and subsidiary libraries. Complexity in this
diff is due to the GNU_RELRO and OPENBSD_MUTABLE sections.
Tested in snaps for about 3 weeks, with some bootstrap related pain felt in ports
ok kettenis, much help from others.

dtors were broken by trying to reuse DF_1_NODELETE to hint that this
library would never unload, and could be immutable. Pass a seperate
flag for our purposes
Noticed from regress tests by anton, ok kettenis

Support RTLD_NOLOAD in ld.so. From guenther@. OK jca@ guenther@

Prep .c files for removing the #includes from */archdep.h
* replace #include "archdep.h" with #includes of what is used, pulling in
"syscall.h", "util.h", and "archdep.h" as needed
* delete #include <sys/syscall.h> from syscall.h
* only pull in <sys/stat.h> to the three files that use _dl_fstat(),
forward declare struct stat in syscall.h for the others
* NBBY is for <sys/select.h> macros; just use '8' in dl_printf.c
* <machine/vmparam.h> is only needed on i386; conditionalize it
* stop using __LDPGSZ: use _MAX_PAGE_SHIFT (already used by malloc.c)
where necessary
* delete other bogus #includes, order legit per style: <sys/*> then
<*/*>, then <*>, then "*"

dir.c improvement from jsg@
ok and testing assistance deraadt@

On i386 don't attempt to map shared libraries in low memory when
a large executable's .text section crosses the 512MB exec line.

Executables that have MAXTSIZ > 64MB can map above the default
512MB exec line. When this happens, shared libs that attempt to map
into low memory will find their .data section can not be mapped. ld.so
will attempt to remap the share lib at higher addresses until it can be
mapped. For very large executables like chrome this process is very
time consuming. This change detects how much of the executable's
.text section exceeds 512MB and uses that as the initial hint for
shared libs to map into which avoids attempting to map into blocked
memory.

okay deraadt@

When loading a library, mmap(2) may fail. Then everything gets
unmapped and ld.so tries again with different random address layout.
In this case, use the new libc executable address for msyscall(2),
not one from the first try. Fixes sporadic bogus syscall on i386.
OK deraadt@

print addresses upon msyscall failure, for now

Sigh, fix i386 msyscall() case to permission the correct address range.

Repurpose the "syscalls must be on a writeable page" mechanism to
enforce a new policy: system calls must be in pre-registered regions.
We have discussed more strict checks than this, but none satisfy the
cost/benefit based upon our understanding of attack methods, anyways
let's see what the next iteration looks like.

This is intended to harden (translation: attackers must put extra
effort into attacking) against a mixture of W^X failures and JIT bugs
which allow syscall misinterpretation, especially in environments with
polymorphic-instruction/variable-sized instructions. It fits in a bit
with libc/libcrypto/ld.so random relink on boot and no-restart-at-crash
behaviour, particularily for remote problems. Less effective once on-host
since someone the libraries can be read.

For static-executables the kernel registers the main program's
PIE-mapped exec section valid, as well as the randomly-placed sigtramp
page. For dynamic executables ELF ld.so's exec segment is also
labelled valid; ld.so then has enough information to register libc's
exec section as valid via call-once msyscall(2)

For dynamic binaries, we continue to to permit the main program exec
segment because "go" (and potentially a few other applications) have
embedded system calls in the main program. Hopefully at least go gets
fixed soon.

We declare the concept of embedded syscalls a bad idea for numerous
reasons, as we notice the ecosystem has many of
static-syscall-in-base-binary which are dynamically linked against
libraries which in turn use libc, which contains another set of
syscall stubs. We've been concerned about adding even one additional
syscall entry point... but go's approach tends to double the entry-point
attack surface.

This was started at a nano-hackathon in Bob Beck's basement 2 weeks
ago during a long discussion with mortimer trying to hide from the SSL
scream-conversations, and finished in more comfortable circumstances
next to a wood-stove at Elk Lakes cabin with UVM scream-conversations.

ok guenther kettenis mortimer, lots of feedback from others
conversations about go with jsing tb sthen

Convert the child_list member from a linked list to a vector.

ok mpi@

On i386, ensure that the first PT_LOAD segment is below the W^X
line unless it is writable.

lld places read-only sections below the gap so this is needed
to be able to retain W^X with lld. Note however the read-only
sections below the W^X line are now executable on pre-NX
machines and a possible source of gadgets. This is a change
from Gnu ld where RO sections were ordered above the W^X line
and not executable.

okay drahn@ kettenis@ deraadt@

Everyone knows this as ld.so, nor by the ancient name rtld.
ok guenther

On fatal errors, kill ourselves with thrkill(0,9,NULL) instead of
simply exiting, via helper functions _dl_die(), _dl_diedie(), and
_dl_oom().

prompted by a complaint from jsing@
ok jsing@ deraadt@

the slimmed down random functions inside ld.so are strict clones of the
libc arc4random API, so call them _dl_{arc4random,arcrandombuf}
ok tedu guenther

Look for a PT_GNU_RELRO section per object and, if present, mprotect that
range instead of the [__got_start, __got_end) range.
On many archs this will cover _DYNAMIC too, so move up the DT_DEBUG handling
to before relocations and the mprotect are done.

ok kettenis@

Remove prebind support: binding to symbol table indices is too fragile
for our development process.

ok kettenis@ deraadt@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Export environ and __progname, making the latter a copy of just the filename
portion like crt0 does. This is prep for eliminating _dl_fixup_user_env()
Mark almost everything in resolve.h as hidden, to improve code generation.

ok kettenis@ mpi@ "good time" deraadt@

assign pointers to NULL rather than 0

Fix unloading of load groups when the last reference wasn't on the
load_object but rather some descendent. Detect that case in
_dl_unload_shlib() and switch to unloading the entire group.

Based on partial analyses by Henri Kemppainen (duclare (at) guu.fi)
and Peter Hajdu (peter.ferenc.hajdu (at) gmail.com)
ok millert@

delete annoying white space that annoy

<sys/param.h> to <limits.h> conversion. Verified binaries
ok millert, thanks to doug for process advice

check all memory allocations; ok miod@ guenther@

Use O_CLOEXEC to make sure fork+exec in a threaded process can't
see the fds used by dlopen()

ok otto@ miod@

Move to a non-zeroing _dl_malloc, a _dl_calloc and _dl_reallocarry and
fix _dl_strdup to return NULL instead of crash; ok deraadt@

Add support for .openbsd.randomdata sections and PT_OPENBSD_RANDOMIZE
segments to the kernel, ld (2.15), and ld.so. Tested on alpha, amd64,
i386, macppc, and sparc64 (thanks naddy, mpi, and okan!).

Idea discussed for some time; committing now for further testing.
ok deraadt

Add a new mmap(2) flag __MAP_NOREMAP for use with MAP_FIXED to
indicate that the kernel should fail with MAP_FAILED if the specified
address is not currently available instead of unmapping it.

Change ld.so on i386 to make use of __MAP_NOREMAP to improve
reliability.

__MAP_NOREMAP diff by guenther based on an earlier diff by Ariane;
ld.so bits by guenther and me
bulk build stress testing of earlier diffs by sthen
ok deraadt; committing now for further testing

Fix loaded object sod matching: when we load libfoo.so.X.Y into
memory, we should be able to match other requests for libfoo.so.X.Z
against that same object.

ok kurt, kettenis

Refuse to load ELF objects that contain a PT_TLS program header.
Otherwise the binary assumes that the requested TLS storage has been
allocated and will happily use it, resulting in unwanted memory corruption.

ok guenther@

Don't mmap 0 byte areas, treat them as a noop instead.

ok miod@

Add support for getting some flags from DT_FLAGS_1: new flags
DF_1_NODELETE and DF_1_INITFIRST, as well as DF_1_NOW and DF_1_GLOBAL.

Committing for kurt@ who worked out the final version; ok guenther@ drahn@

Fix error message when ld.so ends up loading a different than expected
library, but other library needs the one loaded. mostly ok kurt@

Search loaded libs first and add support for SONAME matching. ok drahn@

Fix mmap() error checking to be correct 64-bit addresses. Consistently
use _dl_mmap_error() to check for mmap() errors. Adjust datatypes of
some local vars for 64-bit safety.

okay millert@ drahn@

Constantly fill in the program header pointer and count in elf_object_t
for all objects which simplifies phdr usage in a few places.
"go for it" drahn@

pretty this up; ok drahn

prebind - how to prelink a binary without throwing security out the window

Prelink fixes the address of libraries making 'return to libc' attacks trival,
prebind uses a different method to achieve most of the same gains, however
without adding any security conerns.

Still under development, now in-tree.

add RTLD_NOW support to dlopen and propogate -z now to dep libs.
ok drahn@

Split grpsym_list creation away from child_list creation and change
grpsym_list order to match Sun's docs. Also corrects bugs where
grpsym_list was either not created or partially created.

introduce object ref count macros (suggested by dale). no functional
change.

separate load group references from dep lib child/dload lists. move load
group refs to own per object ref counter (grprefcount) and list
(grpref_list). corrects more complex load group ref cases and side effects
from initial implementation. design ideas and ok drahn@

refcount corrections: count common dep libs once and centralize dep lib
refcount increments to _dl_link_sub. adjust _dl_notify_unload_shlib to
match new refcount method. ok drahn@

handle references to load groups caused by dlopen()ing of depenant
members of the load group. work by kurt@ and myself

correct last commit, in both files.

increment refcount when opening a lib that is already open (dev/inode
case) ok drahn@

Fully unload dependant libraries, fixes gphoto2 bug.

Rework symbol lookup to more closely match sun's documentation, now
treats dlopens as load groups. ok kurt@

removed unused var, pointed out by wilfried@

unload is not to be recursive anymore, previous diff change method but
this code was missed, fixes issue seen by tedu and millert hopefully php too.

fixes for dlclose, ok kettenis@

Recommit the destructor order fix, now that the amd64 bug was fixed.
'no problem' pval@

backout -- breaks at least amd64; spotted by marc

Do a better job of running destructors in the right order.

Code reorganization, move copied code in library.c and library_mquery.c
into its own file. no functional change.

fix PR3944 on i386 (ugh), from library.c
--
Fix some problems related to LD_LIBRARY_PATH parsing where it would not
correctly deal with current directory searches specified by "::", ":foo" or
"foo:"
--
ok mickey@

Fix an issue where a shared library could be loaded at two different
locations by resolving all dlopens back to a dev/inode.

i.e. Don't load a library if the dev/inode it stats back to matches one
already in our list.

fix started (and ok'ed) drahn@. ok deraadt@.
"doesn't break anything yet" pval@ art@ brad@

Fix PR 3371, symbol lookup in dlopen()ed objects is not correct. Correct
behavior for RTLD_GLOBAL/RTLD_LOCAL is now supported. ok espie@

Fix print if minor of library used is is less than requested. ok tdeval@

various proto, ansi, and knf repair. tested on all architectures that
use it. (build may require make cleandir because of .depend balony)

Correct library search algorithm, wrt versioned objects

Dynamic linking random order fixes. This enables random library ordering.
Tested by naddy@ and others.

pefo 3/4 licence cleanups

When loading a shared object or libraries dependant object, load them
in random order. This will reduce the possiblity of a buffer overflow
being able to predict the addresss of useful code. Can be disabled
with the LD_NORANDOM environment variable for debugging purposes.
ok deraadt.

Change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built, booted, and 'make includes' before building
ld.so with this change.

backout mquery change, something broke when not combined with a different diff.

change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built and installed before building ld.so with this change.
ok millert@ tedu@

Fix the load_size which is used by ldd to print end address.

Adjusting the start is wrong. The value returned by mquery() should be used
here. Fixes a bug with emacs21. tested by marc@ and others.

mquery support, currently for i386 only.

whitespaces

Skip the _dl_msyscall() invocation if tracing library loading.

Problem noted by gnezdo@
ok millert@

validate alignment of ELF program headers

Accumulate intermediate imutables locally before applying

OK deraadt

The next step for mimmutable(). ld.so figures out what regions of memory
of startup shared library mappings can be made immutable, and also does
this for dlope() RTLD_NODELETE and subsidiary libraries. Complexity in this
diff is due to the GNU_RELRO and OPENBSD_MUTABLE sections.
Tested in snaps for about 3 weeks, with some bootstrap related pain felt in ports
ok kettenis, much help from others.

dtors were broken by trying to reuse DF_1_NODELETE to hint that this
library would never unload, and could be immutable. Pass a seperate
flag for our purposes
Noticed from regress tests by anton, ok kettenis

Support RTLD_NOLOAD in ld.so. From guenther@. OK jca@ guenther@

Prep .c files for removing the #includes from */archdep.h
* replace #include "archdep.h" with #includes of what is used, pulling in
"syscall.h", "util.h", and "archdep.h" as needed
* delete #include <sys/syscall.h> from syscall.h
* only pull in <sys/stat.h> to the three files that use _dl_fstat(),
forward declare struct stat in syscall.h for the others
* NBBY is for <sys/select.h> macros; just use '8' in dl_printf.c
* <machine/vmparam.h> is only needed on i386; conditionalize it
* stop using __LDPGSZ: use _MAX_PAGE_SHIFT (already used by malloc.c)
where necessary
* delete other bogus #includes, order legit per style: <sys/*> then
<*/*>, then <*>, then "*"

dir.c improvement from jsg@
ok and testing assistance deraadt@

On i386 don't attempt to map shared libraries in low memory when
a large executable's .text section crosses the 512MB exec line.

Executables that have MAXTSIZ > 64MB can map above the default
512MB exec line. When this happens, shared libs that attempt to map
into low memory will find their .data section can not be mapped. ld.so
will attempt to remap the share lib at higher addresses until it can be
mapped. For very large executables like chrome this process is very
time consuming. This change detects how much of the executable's
.text section exceeds 512MB and uses that as the initial hint for
shared libs to map into which avoids attempting to map into blocked
memory.

okay deraadt@

When loading a library, mmap(2) may fail. Then everything gets
unmapped and ld.so tries again with different random address layout.
In this case, use the new libc executable address for msyscall(2),
not one from the first try. Fixes sporadic bogus syscall on i386.
OK deraadt@

print addresses upon msyscall failure, for now

Sigh, fix i386 msyscall() case to permission the correct address range.

Repurpose the "syscalls must be on a writeable page" mechanism to
enforce a new policy: system calls must be in pre-registered regions.
We have discussed more strict checks than this, but none satisfy the
cost/benefit based upon our understanding of attack methods, anyways
let's see what the next iteration looks like.

This is intended to harden (translation: attackers must put extra
effort into attacking) against a mixture of W^X failures and JIT bugs
which allow syscall misinterpretation, especially in environments with
polymorphic-instruction/variable-sized instructions. It fits in a bit
with libc/libcrypto/ld.so random relink on boot and no-restart-at-crash
behaviour, particularily for remote problems. Less effective once on-host
since someone the libraries can be read.

For static-executables the kernel registers the main program's
PIE-mapped exec section valid, as well as the randomly-placed sigtramp
page. For dynamic executables ELF ld.so's exec segment is also
labelled valid; ld.so then has enough information to register libc's
exec section as valid via call-once msyscall(2)

For dynamic binaries, we continue to to permit the main program exec
segment because "go" (and potentially a few other applications) have
embedded system calls in the main program. Hopefully at least go gets
fixed soon.

We declare the concept of embedded syscalls a bad idea for numerous
reasons, as we notice the ecosystem has many of
static-syscall-in-base-binary which are dynamically linked against
libraries which in turn use libc, which contains another set of
syscall stubs. We've been concerned about adding even one additional
syscall entry point... but go's approach tends to double the entry-point
attack surface.

This was started at a nano-hackathon in Bob Beck's basement 2 weeks
ago during a long discussion with mortimer trying to hide from the SSL
scream-conversations, and finished in more comfortable circumstances
next to a wood-stove at Elk Lakes cabin with UVM scream-conversations.

ok guenther kettenis mortimer, lots of feedback from others
conversations about go with jsing tb sthen

Convert the child_list member from a linked list to a vector.

ok mpi@

On i386, ensure that the first PT_LOAD segment is below the W^X
line unless it is writable.

lld places read-only sections below the gap so this is needed
to be able to retain W^X with lld. Note however the read-only
sections below the W^X line are now executable on pre-NX
machines and a possible source of gadgets. This is a change
from Gnu ld where RO sections were ordered above the W^X line
and not executable.

okay drahn@ kettenis@ deraadt@

Everyone knows this as ld.so, nor by the ancient name rtld.
ok guenther

On fatal errors, kill ourselves with thrkill(0,9,NULL) instead of
simply exiting, via helper functions _dl_die(), _dl_diedie(), and
_dl_oom().

prompted by a complaint from jsing@
ok jsing@ deraadt@

the slimmed down random functions inside ld.so are strict clones of the
libc arc4random API, so call them _dl_{arc4random,arcrandombuf}
ok tedu guenther

Look for a PT_GNU_RELRO section per object and, if present, mprotect that
range instead of the [__got_start, __got_end) range.
On many archs this will cover _DYNAMIC too, so move up the DT_DEBUG handling
to before relocations and the mprotect are done.

ok kettenis@

Remove prebind support: binding to symbol table indices is too fragile
for our development process.

ok kettenis@ deraadt@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Export environ and __progname, making the latter a copy of just the filename
portion like crt0 does. This is prep for eliminating _dl_fixup_user_env()
Mark almost everything in resolve.h as hidden, to improve code generation.

ok kettenis@ mpi@ "good time" deraadt@

assign pointers to NULL rather than 0

Fix unloading of load groups when the last reference wasn't on the
load_object but rather some descendent. Detect that case in
_dl_unload_shlib() and switch to unloading the entire group.

Based on partial analyses by Henri Kemppainen (duclare (at) guu.fi)
and Peter Hajdu (peter.ferenc.hajdu (at) gmail.com)
ok millert@

delete annoying white space that annoy

<sys/param.h> to <limits.h> conversion. Verified binaries
ok millert, thanks to doug for process advice

check all memory allocations; ok miod@ guenther@

Use O_CLOEXEC to make sure fork+exec in a threaded process can't
see the fds used by dlopen()

ok otto@ miod@

Move to a non-zeroing _dl_malloc, a _dl_calloc and _dl_reallocarry and
fix _dl_strdup to return NULL instead of crash; ok deraadt@

Add support for .openbsd.randomdata sections and PT_OPENBSD_RANDOMIZE
segments to the kernel, ld (2.15), and ld.so. Tested on alpha, amd64,
i386, macppc, and sparc64 (thanks naddy, mpi, and okan!).

Idea discussed for some time; committing now for further testing.
ok deraadt

Add a new mmap(2) flag __MAP_NOREMAP for use with MAP_FIXED to
indicate that the kernel should fail with MAP_FAILED if the specified
address is not currently available instead of unmapping it.

Change ld.so on i386 to make use of __MAP_NOREMAP to improve
reliability.

__MAP_NOREMAP diff by guenther based on an earlier diff by Ariane;
ld.so bits by guenther and me
bulk build stress testing of earlier diffs by sthen
ok deraadt; committing now for further testing

Fix loaded object sod matching: when we load libfoo.so.X.Y into
memory, we should be able to match other requests for libfoo.so.X.Z
against that same object.

ok kurt, kettenis

Refuse to load ELF objects that contain a PT_TLS program header.
Otherwise the binary assumes that the requested TLS storage has been
allocated and will happily use it, resulting in unwanted memory corruption.

ok guenther@

Don't mmap 0 byte areas, treat them as a noop instead.

ok miod@

Add support for getting some flags from DT_FLAGS_1: new flags
DF_1_NODELETE and DF_1_INITFIRST, as well as DF_1_NOW and DF_1_GLOBAL.

Committing for kurt@ who worked out the final version; ok guenther@ drahn@

Fix error message when ld.so ends up loading a different than expected
library, but other library needs the one loaded. mostly ok kurt@

Search loaded libs first and add support for SONAME matching. ok drahn@

Fix mmap() error checking to be correct 64-bit addresses. Consistently
use _dl_mmap_error() to check for mmap() errors. Adjust datatypes of
some local vars for 64-bit safety.

okay millert@ drahn@

Constantly fill in the program header pointer and count in elf_object_t
for all objects which simplifies phdr usage in a few places.
"go for it" drahn@

pretty this up; ok drahn

prebind - how to prelink a binary without throwing security out the window

Prelink fixes the address of libraries making 'return to libc' attacks trival,
prebind uses a different method to achieve most of the same gains, however
without adding any security conerns.

Still under development, now in-tree.

add RTLD_NOW support to dlopen and propogate -z now to dep libs.
ok drahn@

Split grpsym_list creation away from child_list creation and change
grpsym_list order to match Sun's docs. Also corrects bugs where
grpsym_list was either not created or partially created.

introduce object ref count macros (suggested by dale). no functional
change.

separate load group references from dep lib child/dload lists. move load
group refs to own per object ref counter (grprefcount) and list
(grpref_list). corrects more complex load group ref cases and side effects
from initial implementation. design ideas and ok drahn@

refcount corrections: count common dep libs once and centralize dep lib
refcount increments to _dl_link_sub. adjust _dl_notify_unload_shlib to
match new refcount method. ok drahn@

handle references to load groups caused by dlopen()ing of depenant
members of the load group. work by kurt@ and myself

correct last commit, in both files.

increment refcount when opening a lib that is already open (dev/inode
case) ok drahn@

Fully unload dependant libraries, fixes gphoto2 bug.

Rework symbol lookup to more closely match sun's documentation, now
treats dlopens as load groups. ok kurt@

removed unused var, pointed out by wilfried@

unload is not to be recursive anymore, previous diff change method but
this code was missed, fixes issue seen by tedu and millert hopefully php too.

fixes for dlclose, ok kettenis@

Recommit the destructor order fix, now that the amd64 bug was fixed.
'no problem' pval@

backout -- breaks at least amd64; spotted by marc

Do a better job of running destructors in the right order.

Code reorganization, move copied code in library.c and library_mquery.c
into its own file. no functional change.

fix PR3944 on i386 (ugh), from library.c
--
Fix some problems related to LD_LIBRARY_PATH parsing where it would not
correctly deal with current directory searches specified by "::", ":foo" or
"foo:"
--
ok mickey@

Fix an issue where a shared library could be loaded at two different
locations by resolving all dlopens back to a dev/inode.

i.e. Don't load a library if the dev/inode it stats back to matches one
already in our list.

fix started (and ok'ed) drahn@. ok deraadt@.
"doesn't break anything yet" pval@ art@ brad@

Fix PR 3371, symbol lookup in dlopen()ed objects is not correct. Correct
behavior for RTLD_GLOBAL/RTLD_LOCAL is now supported. ok espie@

Fix print if minor of library used is is less than requested. ok tdeval@

various proto, ansi, and knf repair. tested on all architectures that
use it. (build may require make cleandir because of .depend balony)

Correct library search algorithm, wrt versioned objects

Dynamic linking random order fixes. This enables random library ordering.
Tested by naddy@ and others.

pefo 3/4 licence cleanups

When loading a shared object or libraries dependant object, load them
in random order. This will reduce the possiblity of a buffer overflow
being able to predict the addresss of useful code. Can be disabled
with the LD_NORANDOM environment variable for debugging purposes.
ok deraadt.

Change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built, booted, and 'make includes' before building
ld.so with this change.

backout mquery change, something broke when not combined with a different diff.

change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built and installed before building ld.so with this change.
ok millert@ tedu@

Fix the load_size which is used by ldd to print end address.

Adjusting the start is wrong. The value returned by mquery() should be used
here. Fixes a bug with emacs21. tested by marc@ and others.

mquery support, currently for i386 only.

Skip the _dl_msyscall() invocation if tracing library loading.

Problem noted by gnezdo@
ok millert@

validate alignment of ELF program headers

Accumulate intermediate imutables locally before applying

OK deraadt

The next step for mimmutable(). ld.so figures out what regions of memory
of startup shared library mappings can be made immutable, and also does
this for dlope() RTLD_NODELETE and subsidiary libraries. Complexity in this
diff is due to the GNU_RELRO and OPENBSD_MUTABLE sections.
Tested in snaps for about 3 weeks, with some bootstrap related pain felt in ports
ok kettenis, much help from others.

dtors were broken by trying to reuse DF_1_NODELETE to hint that this
library would never unload, and could be immutable. Pass a seperate
flag for our purposes
Noticed from regress tests by anton, ok kettenis

Support RTLD_NOLOAD in ld.so. From guenther@. OK jca@ guenther@

Prep .c files for removing the #includes from */archdep.h
* replace #include "archdep.h" with #includes of what is used, pulling in
"syscall.h", "util.h", and "archdep.h" as needed
* delete #include <sys/syscall.h> from syscall.h
* only pull in <sys/stat.h> to the three files that use _dl_fstat(),
forward declare struct stat in syscall.h for the others
* NBBY is for <sys/select.h> macros; just use '8' in dl_printf.c
* <machine/vmparam.h> is only needed on i386; conditionalize it
* stop using __LDPGSZ: use _MAX_PAGE_SHIFT (already used by malloc.c)
where necessary
* delete other bogus #includes, order legit per style: <sys/*> then
<*/*>, then <*>, then "*"

dir.c improvement from jsg@
ok and testing assistance deraadt@

On i386 don't attempt to map shared libraries in low memory when
a large executable's .text section crosses the 512MB exec line.

Executables that have MAXTSIZ > 64MB can map above the default
512MB exec line. When this happens, shared libs that attempt to map
into low memory will find their .data section can not be mapped. ld.so
will attempt to remap the share lib at higher addresses until it can be
mapped. For very large executables like chrome this process is very
time consuming. This change detects how much of the executable's
.text section exceeds 512MB and uses that as the initial hint for
shared libs to map into which avoids attempting to map into blocked
memory.

okay deraadt@

When loading a library, mmap(2) may fail. Then everything gets
unmapped and ld.so tries again with different random address layout.
In this case, use the new libc executable address for msyscall(2),
not one from the first try. Fixes sporadic bogus syscall on i386.
OK deraadt@

print addresses upon msyscall failure, for now

Sigh, fix i386 msyscall() case to permission the correct address range.

Repurpose the "syscalls must be on a writeable page" mechanism to
enforce a new policy: system calls must be in pre-registered regions.
We have discussed more strict checks than this, but none satisfy the
cost/benefit based upon our understanding of attack methods, anyways
let's see what the next iteration looks like.

This is intended to harden (translation: attackers must put extra
effort into attacking) against a mixture of W^X failures and JIT bugs
which allow syscall misinterpretation, especially in environments with
polymorphic-instruction/variable-sized instructions. It fits in a bit
with libc/libcrypto/ld.so random relink on boot and no-restart-at-crash
behaviour, particularily for remote problems. Less effective once on-host
since someone the libraries can be read.

For static-executables the kernel registers the main program's
PIE-mapped exec section valid, as well as the randomly-placed sigtramp
page. For dynamic executables ELF ld.so's exec segment is also
labelled valid; ld.so then has enough information to register libc's
exec section as valid via call-once msyscall(2)

For dynamic binaries, we continue to to permit the main program exec
segment because "go" (and potentially a few other applications) have
embedded system calls in the main program. Hopefully at least go gets
fixed soon.

We declare the concept of embedded syscalls a bad idea for numerous
reasons, as we notice the ecosystem has many of
static-syscall-in-base-binary which are dynamically linked against
libraries which in turn use libc, which contains another set of
syscall stubs. We've been concerned about adding even one additional
syscall entry point... but go's approach tends to double the entry-point
attack surface.

This was started at a nano-hackathon in Bob Beck's basement 2 weeks
ago during a long discussion with mortimer trying to hide from the SSL
scream-conversations, and finished in more comfortable circumstances
next to a wood-stove at Elk Lakes cabin with UVM scream-conversations.

ok guenther kettenis mortimer, lots of feedback from others
conversations about go with jsing tb sthen

Convert the child_list member from a linked list to a vector.

ok mpi@

On i386, ensure that the first PT_LOAD segment is below the W^X
line unless it is writable.

lld places read-only sections below the gap so this is needed
to be able to retain W^X with lld. Note however the read-only
sections below the W^X line are now executable on pre-NX
machines and a possible source of gadgets. This is a change
from Gnu ld where RO sections were ordered above the W^X line
and not executable.

okay drahn@ kettenis@ deraadt@

Everyone knows this as ld.so, nor by the ancient name rtld.
ok guenther

On fatal errors, kill ourselves with thrkill(0,9,NULL) instead of
simply exiting, via helper functions _dl_die(), _dl_diedie(), and
_dl_oom().

prompted by a complaint from jsing@
ok jsing@ deraadt@

the slimmed down random functions inside ld.so are strict clones of the
libc arc4random API, so call them _dl_{arc4random,arcrandombuf}
ok tedu guenther

Look for a PT_GNU_RELRO section per object and, if present, mprotect that
range instead of the [__got_start, __got_end) range.
On many archs this will cover _DYNAMIC too, so move up the DT_DEBUG handling
to before relocations and the mprotect are done.

ok kettenis@

Remove prebind support: binding to symbol table indices is too fragile
for our development process.

ok kettenis@ deraadt@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Export environ and __progname, making the latter a copy of just the filename
portion like crt0 does. This is prep for eliminating _dl_fixup_user_env()
Mark almost everything in resolve.h as hidden, to improve code generation.

ok kettenis@ mpi@ "good time" deraadt@

assign pointers to NULL rather than 0

Fix unloading of load groups when the last reference wasn't on the
load_object but rather some descendent. Detect that case in
_dl_unload_shlib() and switch to unloading the entire group.

Based on partial analyses by Henri Kemppainen (duclare (at) guu.fi)
and Peter Hajdu (peter.ferenc.hajdu (at) gmail.com)
ok millert@

delete annoying white space that annoy

<sys/param.h> to <limits.h> conversion. Verified binaries
ok millert, thanks to doug for process advice

check all memory allocations; ok miod@ guenther@

Use O_CLOEXEC to make sure fork+exec in a threaded process can't
see the fds used by dlopen()

ok otto@ miod@

Move to a non-zeroing _dl_malloc, a _dl_calloc and _dl_reallocarry and
fix _dl_strdup to return NULL instead of crash; ok deraadt@

Add support for .openbsd.randomdata sections and PT_OPENBSD_RANDOMIZE
segments to the kernel, ld (2.15), and ld.so. Tested on alpha, amd64,
i386, macppc, and sparc64 (thanks naddy, mpi, and okan!).

Idea discussed for some time; committing now for further testing.
ok deraadt

Add a new mmap(2) flag __MAP_NOREMAP for use with MAP_FIXED to
indicate that the kernel should fail with MAP_FAILED if the specified
address is not currently available instead of unmapping it.

Change ld.so on i386 to make use of __MAP_NOREMAP to improve
reliability.

__MAP_NOREMAP diff by guenther based on an earlier diff by Ariane;
ld.so bits by guenther and me
bulk build stress testing of earlier diffs by sthen
ok deraadt; committing now for further testing

Fix loaded object sod matching: when we load libfoo.so.X.Y into
memory, we should be able to match other requests for libfoo.so.X.Z
against that same object.

ok kurt, kettenis

Refuse to load ELF objects that contain a PT_TLS program header.
Otherwise the binary assumes that the requested TLS storage has been
allocated and will happily use it, resulting in unwanted memory corruption.

ok guenther@

Don't mmap 0 byte areas, treat them as a noop instead.

ok miod@

Add support for getting some flags from DT_FLAGS_1: new flags
DF_1_NODELETE and DF_1_INITFIRST, as well as DF_1_NOW and DF_1_GLOBAL.

Committing for kurt@ who worked out the final version; ok guenther@ drahn@

Fix error message when ld.so ends up loading a different than expected
library, but other library needs the one loaded. mostly ok kurt@

Search loaded libs first and add support for SONAME matching. ok drahn@

Fix mmap() error checking to be correct 64-bit addresses. Consistently
use _dl_mmap_error() to check for mmap() errors. Adjust datatypes of
some local vars for 64-bit safety.

okay millert@ drahn@

Constantly fill in the program header pointer and count in elf_object_t
for all objects which simplifies phdr usage in a few places.
"go for it" drahn@

pretty this up; ok drahn

prebind - how to prelink a binary without throwing security out the window

Prelink fixes the address of libraries making 'return to libc' attacks trival,
prebind uses a different method to achieve most of the same gains, however
without adding any security conerns.

Still under development, now in-tree.

add RTLD_NOW support to dlopen and propogate -z now to dep libs.
ok drahn@

Split grpsym_list creation away from child_list creation and change
grpsym_list order to match Sun's docs. Also corrects bugs where
grpsym_list was either not created or partially created.

introduce object ref count macros (suggested by dale). no functional
change.

separate load group references from dep lib child/dload lists. move load
group refs to own per object ref counter (grprefcount) and list
(grpref_list). corrects more complex load group ref cases and side effects
from initial implementation. design ideas and ok drahn@

refcount corrections: count common dep libs once and centralize dep lib
refcount increments to _dl_link_sub. adjust _dl_notify_unload_shlib to
match new refcount method. ok drahn@

handle references to load groups caused by dlopen()ing of depenant
members of the load group. work by kurt@ and myself

correct last commit, in both files.

increment refcount when opening a lib that is already open (dev/inode
case) ok drahn@

Fully unload dependant libraries, fixes gphoto2 bug.

Rework symbol lookup to more closely match sun's documentation, now
treats dlopens as load groups. ok kurt@

removed unused var, pointed out by wilfried@

unload is not to be recursive anymore, previous diff change method but
this code was missed, fixes issue seen by tedu and millert hopefully php too.

fixes for dlclose, ok kettenis@

Recommit the destructor order fix, now that the amd64 bug was fixed.
'no problem' pval@

backout -- breaks at least amd64; spotted by marc

Do a better job of running destructors in the right order.

Code reorganization, move copied code in library.c and library_mquery.c
into its own file. no functional change.

fix PR3944 on i386 (ugh), from library.c
--
Fix some problems related to LD_LIBRARY_PATH parsing where it would not
correctly deal with current directory searches specified by "::", ":foo" or
"foo:"
--
ok mickey@

Fix an issue where a shared library could be loaded at two different
locations by resolving all dlopens back to a dev/inode.

i.e. Don't load a library if the dev/inode it stats back to matches one
already in our list.

fix started (and ok'ed) drahn@. ok deraadt@.
"doesn't break anything yet" pval@ art@ brad@

Fix PR 3371, symbol lookup in dlopen()ed objects is not correct. Correct
behavior for RTLD_GLOBAL/RTLD_LOCAL is now supported. ok espie@

Fix print if minor of library used is is less than requested. ok tdeval@

various proto, ansi, and knf repair. tested on all architectures that
use it. (build may require make cleandir because of .depend balony)

Correct library search algorithm, wrt versioned objects

Dynamic linking random order fixes. This enables random library ordering.
Tested by naddy@ and others.

pefo 3/4 licence cleanups

When loading a shared object or libraries dependant object, load them
in random order. This will reduce the possiblity of a buffer overflow
being able to predict the addresss of useful code. Can be disabled
with the LD_NORANDOM environment variable for debugging purposes.
ok deraadt.

Change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built, booted, and 'make includes' before building
ld.so with this change.

backout mquery change, something broke when not combined with a different diff.

change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built and installed before building ld.so with this change.
ok millert@ tedu@

Fix the load_size which is used by ldd to print end address.

Adjusting the start is wrong. The value returned by mquery() should be used
here. Fixes a bug with emacs21. tested by marc@ and others.

mquery support, currently for i386 only.

validate alignment of ELF program headers

Accumulate intermediate imutables locally before applying

OK deraadt

The next step for mimmutable(). ld.so figures out what regions of memory
of startup shared library mappings can be made immutable, and also does
this for dlope() RTLD_NODELETE and subsidiary libraries. Complexity in this
diff is due to the GNU_RELRO and OPENBSD_MUTABLE sections.
Tested in snaps for about 3 weeks, with some bootstrap related pain felt in ports
ok kettenis, much help from others.

dtors were broken by trying to reuse DF_1_NODELETE to hint that this
library would never unload, and could be immutable. Pass a seperate
flag for our purposes
Noticed from regress tests by anton, ok kettenis

Support RTLD_NOLOAD in ld.so. From guenther@. OK jca@ guenther@

Prep .c files for removing the #includes from */archdep.h
* replace #include "archdep.h" with #includes of what is used, pulling in
"syscall.h", "util.h", and "archdep.h" as needed
* delete #include <sys/syscall.h> from syscall.h
* only pull in <sys/stat.h> to the three files that use _dl_fstat(),
forward declare struct stat in syscall.h for the others
* NBBY is for <sys/select.h> macros; just use '8' in dl_printf.c
* <machine/vmparam.h> is only needed on i386; conditionalize it
* stop using __LDPGSZ: use _MAX_PAGE_SHIFT (already used by malloc.c)
where necessary
* delete other bogus #includes, order legit per style: <sys/*> then
<*/*>, then <*>, then "*"

dir.c improvement from jsg@
ok and testing assistance deraadt@

On i386 don't attempt to map shared libraries in low memory when
a large executable's .text section crosses the 512MB exec line.

Executables that have MAXTSIZ > 64MB can map above the default
512MB exec line. When this happens, shared libs that attempt to map
into low memory will find their .data section can not be mapped. ld.so
will attempt to remap the share lib at higher addresses until it can be
mapped. For very large executables like chrome this process is very
time consuming. This change detects how much of the executable's
.text section exceeds 512MB and uses that as the initial hint for
shared libs to map into which avoids attempting to map into blocked
memory.

okay deraadt@

When loading a library, mmap(2) may fail. Then everything gets
unmapped and ld.so tries again with different random address layout.
In this case, use the new libc executable address for msyscall(2),
not one from the first try. Fixes sporadic bogus syscall on i386.
OK deraadt@

print addresses upon msyscall failure, for now

Sigh, fix i386 msyscall() case to permission the correct address range.

Repurpose the "syscalls must be on a writeable page" mechanism to
enforce a new policy: system calls must be in pre-registered regions.
We have discussed more strict checks than this, but none satisfy the
cost/benefit based upon our understanding of attack methods, anyways
let's see what the next iteration looks like.

This is intended to harden (translation: attackers must put extra
effort into attacking) against a mixture of W^X failures and JIT bugs
which allow syscall misinterpretation, especially in environments with
polymorphic-instruction/variable-sized instructions. It fits in a bit
with libc/libcrypto/ld.so random relink on boot and no-restart-at-crash
behaviour, particularily for remote problems. Less effective once on-host
since someone the libraries can be read.

For static-executables the kernel registers the main program's
PIE-mapped exec section valid, as well as the randomly-placed sigtramp
page. For dynamic executables ELF ld.so's exec segment is also
labelled valid; ld.so then has enough information to register libc's
exec section as valid via call-once msyscall(2)

For dynamic binaries, we continue to to permit the main program exec
segment because "go" (and potentially a few other applications) have
embedded system calls in the main program. Hopefully at least go gets
fixed soon.

We declare the concept of embedded syscalls a bad idea for numerous
reasons, as we notice the ecosystem has many of
static-syscall-in-base-binary which are dynamically linked against
libraries which in turn use libc, which contains another set of
syscall stubs. We've been concerned about adding even one additional
syscall entry point... but go's approach tends to double the entry-point
attack surface.

This was started at a nano-hackathon in Bob Beck's basement 2 weeks
ago during a long discussion with mortimer trying to hide from the SSL
scream-conversations, and finished in more comfortable circumstances
next to a wood-stove at Elk Lakes cabin with UVM scream-conversations.

ok guenther kettenis mortimer, lots of feedback from others
conversations about go with jsing tb sthen

Convert the child_list member from a linked list to a vector.

ok mpi@

On i386, ensure that the first PT_LOAD segment is below the W^X
line unless it is writable.

lld places read-only sections below the gap so this is needed
to be able to retain W^X with lld. Note however the read-only
sections below the W^X line are now executable on pre-NX
machines and a possible source of gadgets. This is a change
from Gnu ld where RO sections were ordered above the W^X line
and not executable.

okay drahn@ kettenis@ deraadt@

Everyone knows this as ld.so, nor by the ancient name rtld.
ok guenther

On fatal errors, kill ourselves with thrkill(0,9,NULL) instead of
simply exiting, via helper functions _dl_die(), _dl_diedie(), and
_dl_oom().

prompted by a complaint from jsing@
ok jsing@ deraadt@

the slimmed down random functions inside ld.so are strict clones of the
libc arc4random API, so call them _dl_{arc4random,arcrandombuf}
ok tedu guenther

Look for a PT_GNU_RELRO section per object and, if present, mprotect that
range instead of the [__got_start, __got_end) range.
On many archs this will cover _DYNAMIC too, so move up the DT_DEBUG handling
to before relocations and the mprotect are done.

ok kettenis@

Remove prebind support: binding to symbol table indices is too fragile
for our development process.

ok kettenis@ deraadt@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Export environ and __progname, making the latter a copy of just the filename
portion like crt0 does. This is prep for eliminating _dl_fixup_user_env()
Mark almost everything in resolve.h as hidden, to improve code generation.

ok kettenis@ mpi@ "good time" deraadt@

assign pointers to NULL rather than 0

Fix unloading of load groups when the last reference wasn't on the
load_object but rather some descendent. Detect that case in
_dl_unload_shlib() and switch to unloading the entire group.

Based on partial analyses by Henri Kemppainen (duclare (at) guu.fi)
and Peter Hajdu (peter.ferenc.hajdu (at) gmail.com)
ok millert@

delete annoying white space that annoy

<sys/param.h> to <limits.h> conversion. Verified binaries
ok millert, thanks to doug for process advice

check all memory allocations; ok miod@ guenther@

Use O_CLOEXEC to make sure fork+exec in a threaded process can't
see the fds used by dlopen()

ok otto@ miod@

Move to a non-zeroing _dl_malloc, a _dl_calloc and _dl_reallocarry and
fix _dl_strdup to return NULL instead of crash; ok deraadt@

Add support for .openbsd.randomdata sections and PT_OPENBSD_RANDOMIZE
segments to the kernel, ld (2.15), and ld.so. Tested on alpha, amd64,
i386, macppc, and sparc64 (thanks naddy, mpi, and okan!).

Idea discussed for some time; committing now for further testing.
ok deraadt

Add a new mmap(2) flag __MAP_NOREMAP for use with MAP_FIXED to
indicate that the kernel should fail with MAP_FAILED if the specified
address is not currently available instead of unmapping it.

Change ld.so on i386 to make use of __MAP_NOREMAP to improve
reliability.

__MAP_NOREMAP diff by guenther based on an earlier diff by Ariane;
ld.so bits by guenther and me
bulk build stress testing of earlier diffs by sthen
ok deraadt; committing now for further testing

Fix loaded object sod matching: when we load libfoo.so.X.Y into
memory, we should be able to match other requests for libfoo.so.X.Z
against that same object.

ok kurt, kettenis

Refuse to load ELF objects that contain a PT_TLS program header.
Otherwise the binary assumes that the requested TLS storage has been
allocated and will happily use it, resulting in unwanted memory corruption.

ok guenther@

Don't mmap 0 byte areas, treat them as a noop instead.

ok miod@

Add support for getting some flags from DT_FLAGS_1: new flags
DF_1_NODELETE and DF_1_INITFIRST, as well as DF_1_NOW and DF_1_GLOBAL.

Committing for kurt@ who worked out the final version; ok guenther@ drahn@

Fix error message when ld.so ends up loading a different than expected
library, but other library needs the one loaded. mostly ok kurt@

Search loaded libs first and add support for SONAME matching. ok drahn@

Fix mmap() error checking to be correct 64-bit addresses. Consistently
use _dl_mmap_error() to check for mmap() errors. Adjust datatypes of
some local vars for 64-bit safety.

okay millert@ drahn@

Constantly fill in the program header pointer and count in elf_object_t
for all objects which simplifies phdr usage in a few places.
"go for it" drahn@

pretty this up; ok drahn

prebind - how to prelink a binary without throwing security out the window

Prelink fixes the address of libraries making 'return to libc' attacks trival,
prebind uses a different method to achieve most of the same gains, however
without adding any security conerns.

Still under development, now in-tree.

add RTLD_NOW support to dlopen and propogate -z now to dep libs.
ok drahn@

Split grpsym_list creation away from child_list creation and change
grpsym_list order to match Sun's docs. Also corrects bugs where
grpsym_list was either not created or partially created.

introduce object ref count macros (suggested by dale). no functional
change.

separate load group references from dep lib child/dload lists. move load
group refs to own per object ref counter (grprefcount) and list
(grpref_list). corrects more complex load group ref cases and side effects
from initial implementation. design ideas and ok drahn@

refcount corrections: count common dep libs once and centralize dep lib
refcount increments to _dl_link_sub. adjust _dl_notify_unload_shlib to
match new refcount method. ok drahn@

handle references to load groups caused by dlopen()ing of depenant
members of the load group. work by kurt@ and myself

correct last commit, in both files.

increment refcount when opening a lib that is already open (dev/inode
case) ok drahn@

Fully unload dependant libraries, fixes gphoto2 bug.

Rework symbol lookup to more closely match sun's documentation, now
treats dlopens as load groups. ok kurt@

removed unused var, pointed out by wilfried@

unload is not to be recursive anymore, previous diff change method but
this code was missed, fixes issue seen by tedu and millert hopefully php too.

fixes for dlclose, ok kettenis@

Recommit the destructor order fix, now that the amd64 bug was fixed.
'no problem' pval@

backout -- breaks at least amd64; spotted by marc

Do a better job of running destructors in the right order.

Code reorganization, move copied code in library.c and library_mquery.c
into its own file. no functional change.

fix PR3944 on i386 (ugh), from library.c
--
Fix some problems related to LD_LIBRARY_PATH parsing where it would not
correctly deal with current directory searches specified by "::", ":foo" or
"foo:"
--
ok mickey@

Fix an issue where a shared library could be loaded at two different
locations by resolving all dlopens back to a dev/inode.

i.e. Don't load a library if the dev/inode it stats back to matches one
already in our list.

fix started (and ok'ed) drahn@. ok deraadt@.
"doesn't break anything yet" pval@ art@ brad@

Fix PR 3371, symbol lookup in dlopen()ed objects is not correct. Correct
behavior for RTLD_GLOBAL/RTLD_LOCAL is now supported. ok espie@

Fix print if minor of library used is is less than requested. ok tdeval@

various proto, ansi, and knf repair. tested on all architectures that
use it. (build may require make cleandir because of .depend balony)

Correct library search algorithm, wrt versioned objects

Dynamic linking random order fixes. This enables random library ordering.
Tested by naddy@ and others.

pefo 3/4 licence cleanups

When loading a shared object or libraries dependant object, load them
in random order. This will reduce the possiblity of a buffer overflow
being able to predict the addresss of useful code. Can be disabled
with the LD_NORANDOM environment variable for debugging purposes.
ok deraadt.

Change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built, booted, and 'make includes' before building
ld.so with this change.

backout mquery change, something broke when not combined with a different diff.

change mquery() function call signature to be the same a mmap(). It
needs the prot/flags info and passing the addresses via arg/return allows
it to be traced via ktrace better than an in/out paramter.
This adds a new mquery syscall and renames the old one to omquery.
New kernel _MUST_ be built and installed before building ld.so with this change.
ok millert@ tedu@

Fix the load_size which is used by ldd to print end address.

Adjusting the start is wrong. The value returned by mquery() should be used
here. Fixes a bug with emacs21. tested by marc@ and others.

mquery support, currently for i386 only.

Accumulate intermediate imutables locally before applying

OK deraadt

The next step for mimmutable(). ld.so figures out what regions of memory
of startup shared library mappings can be made immutable, and also does
this for dlope() RTLD_NODELETE and subsidiary libraries. Complexity in this
diff is due to the GNU_RELRO and OPENBSD_MUTABLE sections.
Tested in snaps for about 3 weeks, with some bootstrap related pain felt in ports
ok kettenis, much help from others.

dtors were broken by trying to reuse DF_1_NODELETE to hint that this
library would never unload, and could be immutable. Pass a seperate
flag for our purposes
Noticed from regress tests by anton, ok kettenis

Support RTLD_NOLOAD in ld.so. From guenther@. OK jca@ guenther@

Prep .c files for removing the #includes from */archdep.h
* replace #include "archdep.h" with #includes of what is used, pulling in
"syscall.h", "util.h", and "archdep.h" as needed
* delete #include <sys/syscall.h> from syscall.h
* only pull in <sys/stat.h> to the three files that use _dl_fstat(),
forward declare struct stat in syscall.h for the others
* NBBY is for <sys/select.h> macros; just use '8' in dl_printf.c
* <machine/vmparam.h> is only needed on i386; conditionalize it
* stop using __LDPGSZ: use _MAX_PAGE_SHIFT (already used by malloc.c)
where necessary
* delete other bogus #includes, order legit per style: <sys/*> then
<*/*>, then <*>, then "*"

dir.c improvement from jsg@
ok and testing assistance deraadt@

On i386 don't attempt to map shared libraries in low memory when
a large executable's .text section crosses the 512MB exec line.

Executables that have MAXTSIZ > 64MB can map above the default
512MB exec line. When this happens, shared libs that attempt to map
into low memory will find their .data section can not be mapped. ld.so
will attempt to remap the share lib at higher addresses until it can be
mapped. For very large executables like chrome this process is very
time consuming. This change detects how much of the executable's
.text section exceeds 512MB and uses that as the initial hint for
shared libs to map into which avoids attempting to map into blocked
memory.

okay deraadt@

When loading a library, mmap(2) may fail. Then everything gets
unmapped and ld.so tries again with different random address layout.
In this case, use the new libc executable address for msyscall(2),
not one from the first try. Fixes sporadic bogus syscall on i386.
OK deraadt@

print addresses upon msyscall failure, for now

Sigh, fix i386 msyscall() case to permission the correct address range.

Repurpose the "syscalls must be on a writeable page" mechanism to
enforce a new policy: system calls must be in pre-registered regions.
We have discussed more strict checks than this, but none satisfy the
cost/benefit based upon our understanding of attack methods, anyways
let's see what the next iteration looks like.

This is intended to harden (translation: attackers must put extra
effort into attacking) against a mixture of W^X failures and JIT bugs
which allow syscall misinterpretation, especially in environments with
polymorphic-instruction/variable-sized instructions. It fits in a bit
with libc/libcrypto/ld.so random relink on boot and no-restart-at-crash
behaviour, particularily for remote problems. Less effective once on-host
since someone the libraries can be read.

For static-executables the kernel registers the main program's
PIE-mapped exec section valid, as well as the randomly-placed sigtramp
page. For dynamic executables ELF ld.so's exec segment is also
labelled valid; ld.so then has enough information to register libc's
exec section as valid via call-once msyscall(2)

For dynamic binaries, we continue to to permit the main program exec
segment because "go" (and potentially a few other applications) have
embedded system calls in the main program. Hopefully at least go gets
fixed soon.

We declare the concept of embedded syscalls a bad idea for numerous
reasons, as we notice the ecosystem has many of
static-syscall-in-base-binary which are dynamically linked against
libraries which in turn use libc, which contains another set of
syscall stubs. We've been concerned about adding even one additional
syscall entry point... but go's approach tends to double the entry-point
attack surface.

This was started at a nano-hackathon in Bob Beck's basement 2 weeks
ago during a long discussion with mortimer trying to hide from the SSL
scream-conversations, and finished in more comfortable circumstances
next to a wood-stove at Elk Lakes cabin with UVM scream-conversations.

ok guenther kettenis mortimer, lots of feedback from others
conversations about go with jsing tb sthen

Convert the child_list member from a linked list to a vector.

ok mpi@

On i386, ensure that the first PT_LOAD segment is below the W^X
line unless it is writable.

lld places read-only sections below the gap so this is needed
to be able to retain W^X with lld. Note however the read-only
sections below the W^X line are now executable on pre-NX
machines and a possible source of gadgets. This is a change
from Gnu ld where RO sections were ordered above the W^X line
and not executable.

okay drahn@ kettenis@ deraadt@

Everyone knows this as ld.so, nor by the ancient name rtld.
ok guenther

On fatal errors, kill ourselves with thrkill(0,9,NULL) instead of
simply exiting, via helper functions _dl_die(), _dl_diedie(), and
_dl_oom().

prompted by a complaint from jsing@
ok jsing@ deraadt@

the slimmed down random functions inside ld.so are strict clones of the
libc arc4random API, so call them _dl_{arc4random,arcrandombuf}
ok tedu guenther

Look for a PT_GNU_RELRO section per object and, if present, mprotect that
range instead of the [__got_start, __got_end) range.
On many archs this will cover _DYNAMIC too, so move up the DT_DEBUG handling
to before relocations and the mprotect are done.

ok kettenis@

Remove prebind support: binding to symbol table indices is too fragile
for our development process.

ok kettenis@ deraadt@

Use a Thread Information Block in both single and multi-threaded programs.
This stores errno, the cancelation flags, and related bits for each thread
and is allocated by ld.so or libc.a. This is an ABI break from 5.9-stable!

Make libpthread dlopen'able by moving the cancelation wrappers into libc
and doing locking and fork/errno handling via callbacks that libpthread
registers when it first initializes. 'errno' *must* be declared via
<errno.h> now!

Clean up libpthread's symbol exports like libc.

On powerpc, offset the TIB/TCB/TLS data from the register per the ELF spec.

Testing by various, particularly sthen@ and patrick@
ok kettenis@

Export environ and __progname, making the latter a copy of just the filename
portion like crt0 does. This is prep for eliminating _dl_fixup_user_env()
Mark almost everything in resolve.h as hidden, to improve code generation.

ok kettenis@ mpi@ "good time" deraadt@

assign pointers to NULL rather than 0

Fix unloading of load groups when the last reference wasn't on the
load_object but rather some descendent. Detect that case in
_dl_unload_shlib() and switch to unloading the entire group.

Based on partial analyses by Henri Kemppainen (duclare (at) guu.fi)
and Peter Hajdu (peter.ferenc.hajdu (at) gmail.com)
ok millert@

delete annoying white space that annoy

<sys/param.h> to <limits.h> conversion. Verified binaries
ok millert, thanks to doug for process advice

check all memory allocations; ok miod@ guenther@

Use O_CLOEXEC to make sure fork+exec in a threaded process can't
see the fds used by dlopen()

ok otto@ miod@

Move to a non-zeroing _dl_malloc, a _dl_calloc and _dl_reallocarry and
fix _dl_strdup to return NULL instead of crash; ok deraadt@

Add support for .openbsd.randomdata sections and PT_OPENBSD_RANDOMIZE
segments to the kernel, ld (2.15), and ld.so. Tested on alpha, amd64,
i386, macppc, and sparc64 (thanks naddy, mpi, and okan!).

Idea discussed for some time; committing now for further testing.
ok deraadt

Add a new mmap(2) flag __MAP_NOREMAP for use with MAP_FIXED to
indicate that the kernel should fail with MAP_FAILED if the specified
address is not currently available instead of unmapping it.

Change ld.so on i386 to make use of __MAP_NOREMAP to improve
reliability.

__MAP_NOREMAP diff by guenther based on an earlier diff by Ariane;
ld.so bits by guenther and me
bulk build stress testing of earlier diffs by sthen
ok deraadt; committing now for further testing

Fix loaded object sod matching: when we load libfoo.so.X.Y into
memory, we should be able to match other requests for libfoo.so.X.Z
against that same object.

ok kurt, kettenis

Refuse to load ELF objects that contain a PT_TLS program header.
Otherwise the binary assumes that the requested TLS storage has been
allocated and will happily use it, resulting in unwanted memory corruption.

ok guenther@

Don't mmap 0 byte areas, treat them as a noop instead.

ok miod@

Add support for getting some flags from DT_FLAGS_1: new flags
DF_1_NODELETE and DF_1_INITFIRST, as well as DF_1_NOW and DF_1_GLOBAL.

Committing for kurt@ who worked out the final version; ok guenther@ drahn@

Fix error message when ld.so ends up loading a different than expected
library, but other library needs the one loaded. mostly ok kurt@

Search loaded libs first and add support for SONAME matching. ok drahn@

Fix mmap() error checking to be correct 64-bit addresses. Consistently
use _dl_mmap_error() to check for mmap() errors. Adjust datatypes of
some local vars for 64-bit safety.

okay millert@ drahn@

Constantly fill in the program header pointer and count in elf_object_t
for all objects which simplifies phdr usage in a few places.
"go for it" drahn@

pretty this up; ok drahn