runtime_loader: Implement DT_RUNPATH

DT_RUNPATH is generated by the linker instead of DT_RPATH when using --enable-new-dtags
It seems to be the default when using ld.lld
Normally one difference is LD_LIBRARY_PATH is checked before DT_RUNPATH (and not with DT_RPATH),
but we don't check LD_LIBRARY_PATH at the moment. Checking LIBRARY_PATH isn't an option, because
runtime_loader dosesn't use default paths, test suites would define LIBRARY_PATH empty.
Tested with tcpdump build_matrix.sh script, with clang 17, local libpcap, which requires
--disable-new-dtags on r1beta4.

Change-Id: Iacccde8d20e25ad14c5c548dd8832ea32b67e228
Reviewed-on: https://review.haiku-os.org/c/haiku/+/7539
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>

runtime_loader: dlopen() should respect RPATH of the loading module

Change-Id: Ic58edb53114dfff30cc7188957cd32508fa8bd48
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3798
Reviewed-by: Jérôme Duval <jerome.duval@gmail.com>

libroot: make API version available.

Change-Id: I370d5c8e9b4f076e7f115ca6c87bc7f943d36c17
Reviewed-on: https://review.haiku-os.org/c/haiku/+/1850
Reviewed-by: Rene Gollent <rene@gollent.com>

runtime_loader: Reinit the heap lock after fork.

runtime_loader: differentiate between secondary runtime loaders.

runtime_loader: enable elf32 on x86_64, elf64 on x86.

use x86 as default sSearchPathSubDir in compatibility mode.
use the generic memset/memcpy when x86_64 is the primary arch.

Change-Id: Ib464c308ff97f7ae2482ef4c037de1b1bb2bf61b

runtime_loader: Add imageName and exactMatch to symbol lookup.

Extend the get_nearest_symbol_at_address() private runtime_loader
export to include imageName and exactMatch arguments.

The imageName holds the SONAME of the image, if available, so cannot
neccessarily be extracted from the image path.

Whether or not there was an exact match, i.e. the symbol with its size
contains the address, is now returned in exactMatch.

runtime_loader: support linking TLS symbols from any DSO

Previously TLS_DTPMOD relocation blindly returned ID of the current DSO.
This patch does proper symbol lookup if there is a symbol assigned to the
relocation and uses ID of the DSO in which the symbol is defined.

runtime_loader: add support for ELF based TLS

This patch introduces support of ELF based TLS handling with lazy allocation
and initalization of TLS block for each DSO and thread. The implementation
generally follows the official ABI except that generation counter in dtv
is in fact a pointer to Generation object that contains both generation
counter and size of the dtv. That simplified the implementation a bit, but
could be changed later. The ABI requirements regariding in memory position
of TLS block is not honoured what results in static TLS model being
unsupported. However, that should not be a problem as long as
"executables" in Haiku are in fact shared objects and optimizations which
require specific TLS block in memory layout are not possible anyway.

runtime loader: Add support for $ORIGIN in rpath

Like in Linux it resolves to the directory of the shared object whose
needed library is to be loaded.

runtime loader: Export get_executable_architecture() function

Given a path of an ELF file, it tries to determine its architecture.

Runtime loader: rename compatibilitySubDir to abiSpecificSubDir.

libroot: make all areas executable for old binaries

* If at least one image is either B_HAIKU_ABI_GCC_2_ANCIENT or
B_HAIKU_ABI_GCC_2_BEOS almost all areas are marked as executable.
* B_EXECUTE_AREA and B_STACK_AREA are made public. The former is enforced since
the introduction of DEP and apps need it to correctly set area protection.
The latter is currently needed only to recognize stack areas and fix their
protection in compatibility mode, but may also be useful if an app wants
to use sigaltstack from POSIX API.

commpage: randomize position of commpage

This patch introduces randomization of commpage position. From now on commpage
table contains offsets from begining to of the commpage to the particular
commpage entry. Similary addresses of symbols in ELF memory image "commpage"
are just offsets from the begining of the commpage.

This patch also updates KDL so that commpage entries are recognized and shown
correctly in stack trace. An update of Debugger is yet to be done.

Changed runtime_loader to use elf_* typedefs over Elf32_*.

This means that it will be using ELF64 types on x86_64 rather than
ELF32. The next step for supporting x86_64 is to implement relocations.

Fix dladdr behaviour

* If dladdr can't find an exact match, it returns the nearest symbol
less than the given address.
* If no suitable symbol can be found, but the address is within a
loaded library, dladdr returns the library name and base address.

Signed-off-by: Ingo Weinhold <ingo_weinhold@gmx.de>

* Implemented dladdr() in the runtime loader. This is like a gazillion times
faster than before.
* This also solves a TODO in dladdr(), although I did not use
get_library_symbol() as I didn't quite see how that could fit as the comment
suggested; there is now a new function get_symbol_at_address() for this.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42598 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Merged weak-symbols branch.
* Fixed trivial merge conflict in src/system/libroot/posix/locale/nl_langinfo.cpp
* Fixed gcc 2 compilation of src/system/glue/init_term_dyn.c.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39571 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved most code of the heavy elf.cpp into several smaller source files.
* Some style cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30808 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added private get_image_symbol_etc() that can recursively search for a symbol
(similar to how dlsym() works).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29356 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked undefined symbol resolution in the runtime loader. Got rid of
the per-root-image breadth-first sorted image array. Instead we have a
per-image hook function to resolve the symbols. The default function
uses the sLoadedImages list directly, which is breadth-first sorted
anyway. There's also a BeOS function for old-style symbol resolution
and one for add-ons, which lacks a proper implementation yet (just
uses old-style ATM).
* Made the dl*() functions POSIX compliant:
- dlopen() does no longer use load_add_on(), but loads the object as a
library. It also properly supports a NULL name, now -- the previous
"_APP_" work-around did only work, if this soname was set on the
program (unlikely for programs using this API).
- Implemented RTLD_{GLOBAL,LOCAL}.
- dlsym() looks up symbols properly now, i.e. not just in the given
image, but breadth-first for an actual image or in load order for
the global scope. It also supports the not-quite POSIX RTLD_DEFAULT
and RTLD_NEXT extensions. Our RTLD_NEXT finds more symbols than in
Linux (also in later dlopen()ed libraries), but that should be fine.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28568 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Resolved TODO added by Ingo: the runtime loader now lazily updates its
image IDs when needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27611 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added explicit support for loading executables compiled with the
respectively other gcc version on a Haiku compiled with gcc 2 or gcc 4.
The libraries for such an executable are first searched in "gcc4"
respectively "gcc2" subdirectories of the standard search path
directories. If not found there, we try again with the standard paths.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25532 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The runtime_loader's test_executable() no longer analyzes the
executable permissions of the file by hand. We use _kern_access()
instead, which also handles the root case correctly.
* The user and group arguments of test_executable() aren't needed any
longer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24483 a95241bf-73f2-0310-859d-f6bbb57e9c96

bonefish+axeld:
The runtime loader did not correctly resolve %A correctly with the
actual normalized program path. IOW it would not work correctly with
symlinks to applications that had their own lib directory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23986 a95241bf-73f2-0310-859d-f6bbb57e9c96

Give the runtime loader a chance to reinit its semaphore after fork().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21886 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a mechanism to retrieve a BMessage with eventual error descriptions
for _kern_load_image().
* Added KMessage to the runtime_loader (a bit hacky, though) - it will use
it to deliver the above mentioned functionality.
* load_dependencies() did return the wrong status code in case a library
was missing; now it returns B_MISSING_LIBRARY.
* load_dependencies() will now try to load all dependencies when a report
message is requested; therefore, all missing libraries are listed.
* Renamed uspace_program_args to user_space_program_args.
* The kernel filled in various members of the user_space_program_args structure
unsafely, ie. was not using user_memcpy().
* Renamed some local variables in team.c to better fit our style guide (ie.
uargs to userArgs).
* Changed Tracker to use the new _kern_load_image() variant on Haiku to retrieve
and report all missing libraries. This fixes bug #1324.
* Adapted kernel_cpp.cpp to the runtime loader as well; the latter will now
compile with _LOADER_MODE defined.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21715 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Got rid of sLoadingImages. It was always empty.
* The previous symbol resolution code was incorrect. It would search all
loaded images in the order they had been loaded. Thus an add-on would
possibly see a symbol of an earlier loaded add-on. Now we search
recursively starting with the respective root image (executable or
add-on).
* Added BeOS style symbol resolution and made it the default. A symbol
undefined in an image is only searched in its direct dependencies.
Fixes bug #889 (BeOS apps crashing under Haiku when opening a file
panel).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21290 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Switched to the boot loader's heap implementation.
* This sane heap revealed a bunch of bugs (like sLoadedImageCount not being
maintained that was used for various allocations).
* If the allocation of the image fails, opening the app will now just fail
instead of crashing of overwriting random data (IOW VLC will now load
okay, but cannot load some of its add-ons anymore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@19045 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added a private call to get the dependencies of a loaded image. This will be used
to determine linkage of libnet.so vs. libsocket.so/libbind.so in the libnetwork.so.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@19008 a95241bf-73f2-0310-859d-f6bbb57e9c96

While investigating how our deals with doubled shared libs, I found two
issues:
* Our glue code was broken after all - it allowed Haiku apps to start under
BeOS (and vice versa), but the initialization/termination functions were
called with an invalid image ID - on *both* sides! As it turns out, the
Be glue code did *something* with %ebx, but certainly didn't put the image
ID in there, but just passed it on the stack, as we did before (just in
the wrong order...). Therefore, the arch_call_init_term stuff is not
necessary.
* When unloading add-ons, their termination functions were never called, as
the image (for get_image_symbol()) was already made inaccessible, and
therefore the symbol couldn't be found.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17928 a95241bf-73f2-0310-859d-f6bbb57e9c96

Once libroot's getenv() is available, that one will be used instead of the one built in.
This makes the runtime_loader able to adopt path changes properly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17380 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed runtime loader source files.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16935 a95241bf-73f2-0310-859d-f6bbb57e9c96

runtime_loader: Add imageName and exactMatch to symbol lookup.

Extend the get_nearest_symbol_at_address() private runtime_loader
export to include imageName and exactMatch arguments.

The imageName holds the SONAME of the image, if available, so cannot
neccessarily be extracted from the image path.

Whether or not there was an exact match, i.e. the symbol with its size
contains the address, is now returned in exactMatch.

runtime_loader: support linking TLS symbols from any DSO

Previously TLS_DTPMOD relocation blindly returned ID of the current DSO.
This patch does proper symbol lookup if there is a symbol assigned to the
relocation and uses ID of the DSO in which the symbol is defined.

runtime_loader: add support for ELF based TLS

This patch introduces support of ELF based TLS handling with lazy allocation
and initalization of TLS block for each DSO and thread. The implementation
generally follows the official ABI except that generation counter in dtv
is in fact a pointer to Generation object that contains both generation
counter and size of the dtv. That simplified the implementation a bit, but
could be changed later. The ABI requirements regariding in memory position
of TLS block is not honoured what results in static TLS model being
unsupported. However, that should not be a problem as long as
"executables" in Haiku are in fact shared objects and optimizations which
require specific TLS block in memory layout are not possible anyway.

runtime loader: Add support for $ORIGIN in rpath

Like in Linux it resolves to the directory of the shared object whose
needed library is to be loaded.

runtime loader: Export get_executable_architecture() function

Given a path of an ELF file, it tries to determine its architecture.

Runtime loader: rename compatibilitySubDir to abiSpecificSubDir.

libroot: make all areas executable for old binaries

* If at least one image is either B_HAIKU_ABI_GCC_2_ANCIENT or
B_HAIKU_ABI_GCC_2_BEOS almost all areas are marked as executable.
* B_EXECUTE_AREA and B_STACK_AREA are made public. The former is enforced since
the introduction of DEP and apps need it to correctly set area protection.
The latter is currently needed only to recognize stack areas and fix their
protection in compatibility mode, but may also be useful if an app wants
to use sigaltstack from POSIX API.

commpage: randomize position of commpage

This patch introduces randomization of commpage position. From now on commpage
table contains offsets from begining to of the commpage to the particular
commpage entry. Similary addresses of symbols in ELF memory image "commpage"
are just offsets from the begining of the commpage.

This patch also updates KDL so that commpage entries are recognized and shown
correctly in stack trace. An update of Debugger is yet to be done.

Changed runtime_loader to use elf_* typedefs over Elf32_*.

This means that it will be using ELF64 types on x86_64 rather than
ELF32. The next step for supporting x86_64 is to implement relocations.

Fix dladdr behaviour

* If dladdr can't find an exact match, it returns the nearest symbol
less than the given address.
* If no suitable symbol can be found, but the address is within a
loaded library, dladdr returns the library name and base address.

Signed-off-by: Ingo Weinhold <ingo_weinhold@gmx.de>

* Implemented dladdr() in the runtime loader. This is like a gazillion times
faster than before.
* This also solves a TODO in dladdr(), although I did not use
get_library_symbol() as I didn't quite see how that could fit as the comment
suggested; there is now a new function get_symbol_at_address() for this.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42598 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Merged weak-symbols branch.
* Fixed trivial merge conflict in src/system/libroot/posix/locale/nl_langinfo.cpp
* Fixed gcc 2 compilation of src/system/glue/init_term_dyn.c.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39571 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Moved most code of the heavy elf.cpp into several smaller source files.
* Some style cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30808 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added private get_image_symbol_etc() that can recursively search for a symbol
(similar to how dlsym() works).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29356 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Reworked undefined symbol resolution in the runtime loader. Got rid of
the per-root-image breadth-first sorted image array. Instead we have a
per-image hook function to resolve the symbols. The default function
uses the sLoadedImages list directly, which is breadth-first sorted
anyway. There's also a BeOS function for old-style symbol resolution
and one for add-ons, which lacks a proper implementation yet (just
uses old-style ATM).
* Made the dl*() functions POSIX compliant:
- dlopen() does no longer use load_add_on(), but loads the object as a
library. It also properly supports a NULL name, now -- the previous
"_APP_" work-around did only work, if this soname was set on the
program (unlikely for programs using this API).
- Implemented RTLD_{GLOBAL,LOCAL}.
- dlsym() looks up symbols properly now, i.e. not just in the given
image, but breadth-first for an actual image or in load order for
the global scope. It also supports the not-quite POSIX RTLD_DEFAULT
and RTLD_NEXT extensions. Our RTLD_NEXT finds more symbols than in
Linux (also in later dlopen()ed libraries), but that should be fine.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28568 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Resolved TODO added by Ingo: the runtime loader now lazily updates its
image IDs when needed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27611 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added explicit support for loading executables compiled with the
respectively other gcc version on a Haiku compiled with gcc 2 or gcc 4.
The libraries for such an executable are first searched in "gcc4"
respectively "gcc2" subdirectories of the standard search path
directories. If not found there, we try again with the standard paths.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25532 a95241bf-73f2-0310-859d-f6bbb57e9c96

* The runtime_loader's test_executable() no longer analyzes the
executable permissions of the file by hand. We use _kern_access()
instead, which also handles the root case correctly.
* The user and group arguments of test_executable() aren't needed any
longer.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24483 a95241bf-73f2-0310-859d-f6bbb57e9c96

bonefish+axeld:
The runtime loader did not correctly resolve %A correctly with the
actual normalized program path. IOW it would not work correctly with
symlinks to applications that had their own lib directory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23986 a95241bf-73f2-0310-859d-f6bbb57e9c96

Give the runtime loader a chance to reinit its semaphore after fork().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21886 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Added a mechanism to retrieve a BMessage with eventual error descriptions
for _kern_load_image().
* Added KMessage to the runtime_loader (a bit hacky, though) - it will use
it to deliver the above mentioned functionality.
* load_dependencies() did return the wrong status code in case a library
was missing; now it returns B_MISSING_LIBRARY.
* load_dependencies() will now try to load all dependencies when a report
message is requested; therefore, all missing libraries are listed.
* Renamed uspace_program_args to user_space_program_args.
* The kernel filled in various members of the user_space_program_args structure
unsafely, ie. was not using user_memcpy().
* Renamed some local variables in team.c to better fit our style guide (ie.
uargs to userArgs).
* Changed Tracker to use the new _kern_load_image() variant on Haiku to retrieve
and report all missing libraries. This fixes bug #1324.
* Adapted kernel_cpp.cpp to the runtime loader as well; the latter will now
compile with _LOADER_MODE defined.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21715 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Got rid of sLoadingImages. It was always empty.
* The previous symbol resolution code was incorrect. It would search all
loaded images in the order they had been loaded. Thus an add-on would
possibly see a symbol of an earlier loaded add-on. Now we search
recursively starting with the respective root image (executable or
add-on).
* Added BeOS style symbol resolution and made it the default. A symbol
undefined in an image is only searched in its direct dependencies.
Fixes bug #889 (BeOS apps crashing under Haiku when opening a file
panel).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21290 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Switched to the boot loader's heap implementation.
* This sane heap revealed a bunch of bugs (like sLoadedImageCount not being
maintained that was used for various allocations).
* If the allocation of the image fails, opening the app will now just fail
instead of crashing of overwriting random data (IOW VLC will now load
okay, but cannot load some of its add-ons anymore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@19045 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added a private call to get the dependencies of a loaded image. This will be used
to determine linkage of libnet.so vs. libsocket.so/libbind.so in the libnetwork.so.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@19008 a95241bf-73f2-0310-859d-f6bbb57e9c96

While investigating how our deals with doubled shared libs, I found two
issues:
* Our glue code was broken after all - it allowed Haiku apps to start under
BeOS (and vice versa), but the initialization/termination functions were
called with an invalid image ID - on *both* sides! As it turns out, the
Be glue code did *something* with %ebx, but certainly didn't put the image
ID in there, but just passed it on the stack, as we did before (just in
the wrong order...). Therefore, the arch_call_init_term stuff is not
necessary.
* When unloading add-ons, their termination functions were never called, as
the image (for get_image_symbol()) was already made inaccessible, and
therefore the symbol couldn't be found.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17928 a95241bf-73f2-0310-859d-f6bbb57e9c96

Once libroot's getenv() is available, that one will be used instead of the one built in.
This makes the runtime_loader able to adopt path changes properly.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17380 a95241bf-73f2-0310-859d-f6bbb57e9c96

Renamed runtime loader source files.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16935 a95241bf-73f2-0310-859d-f6bbb57e9c96