dtrace: Add the 'oformat' libdtrace option

This option can be used to specify a format to use in DTrace output.
The following formats are supported:
- json
- xml
- html
- none (default DTrace output)

This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:

- dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
- dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
- dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
- dtrace_set_outfp(FILE *) -- sets the output file for oformat.
- Ensure that oformat is correctly checked in the drop handler and record
processing callbacks.

This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.

Reviewed by: markj
Discussed with: phil
MFC after: 2 months
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D41745

dtrace: remove stray {

Fixes: da81cc6035f8283b6adda1ef466977e8c1c5389e
PR: 269128

dtrace: conditionally load the systrace_linux klds when loading dtrace.

When dtrace starts, it tries to detect if the dtrace klds are loaded,
and if not, it loads them by loading the dtraceall kld. This module
depends on most dtrace modules, including systrace for the native
freebsd and freebsd32 ABIs. However, it does not depend on the
systrace_linux klds, as they in turn depend on the linux ABI klds, and
we don't want to load an ABI module that the user has not explicitly
requested. This can leave a naive user in a state where they think all
syscall providers have been loaded, yet linux ABI syscalls are
"invisible" to dtrace.

To fix this, check to see if the linux ABI modules are loaded. If they
are, then load their systrace klds.

Reviewed by: markj, (emaste & jhb, earlier versions)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D37986

libdtrace: Add kinst support

kinst does not instantiate its probes automatically, it only does so on
demand via an ioctl interface implemented by /dev/kinst. This change
modifies libdtrace to perform that work when the script references the
kinst provider, similar to the way pid provider probes are implemented.

Reviewed by: markj
MFC after: 3 months
Sponsored by: Google, Inc. (GSoC 2022)
Differential Revision: https://reviews.freebsd.org/D36852

dtrace: Add a "regs" variable

This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers. It's not clear what
semantics "regs" should have for them anyway.

This is akin to "uregs", which nominally provides access to the
userspace registers. In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.

Usage example: print the contents of %rdi upon each call to
amd64_syscall():

fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}

Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.

Reviewed by: christos
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36799

libdtrace: Add a missing newline to an error message

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

Do not load dtraceall.ko if dtrace.ko is already loaded.

This was the intent of the existing code, but instead it would
unconditionally load dtraceall.ko because of a stale errno value.

Reported by: pho
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

Disable the D subroutines msgsize() and msgdsize().

They are specific to illumos and the corresponding DIF subroutines are
already disabled on FreeBSD.

Reported by: gnn

Add "jid" and "jailname" variables to DTrace.

These return the jail ID and jail name for the traced process,
respectively, and are analogous to "zonename" on Solaris/illumos.
"zonename" is now aliased to "jailname".

Also add some stress tests for the new variables.

Submitted by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
Reviewed by: dteske (previous version)
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13877

Complete support for dtrace's -x setenv option.

This allows one to override the environment for processes created with
dtrace -c. By default, the environment is inherited.

This support was originally merged from illumos in r249367 but was lost
when the commit was later reverted and then brought back piecemeal.

Reported by: Samuel Lepetit <slepetit@apple.com>
MFC after: 2 weeks

Fix the type signature for sx(9) DTrace subroutines.

MFC after: 1 week

Add a O_CLOEXEC use missed in r323166.

PR: 199810
Reported by: Jukka A. Ukkonen <jau789@gmail.com>
MFC after: 3 days

Use O_CLOEXEC when opening persistent handles in libdtrace.

PR: 199810
Submitted by: jau@iki.fi
MFC after: 1 week

Fix memory leaks in error cases in libdtrace.

Submitted by: Tom Rix <trix@juniper.net>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D9705

Remove the DTrace printt and typeref actions.

These are FreeBSD-specific and were added in r178576 to provide the ability
to pretty-print instances of compound types. However, the print action has
long since been augmented to provide this functionality with a simpler
interface.

Discussed with: gnn
Differential Revision: https://reviews.freebsd.org/D8478

libdtrace: allow D libraries to declare dependencies on kernel modules

The "depends_on module" pragma can be used to declare a dependency on a
DTrace module, which for kernel probes corresponds to a KLD. Such
dependencies cannot be checked if the KLD is compiled into the kernel.
Therefore, allow a module dependency to be satisfied if either a kernel
module or a KLD with the specified name is loaded.

Differential Revision: https://reviews.freebsd.org/D2653
Reviewed by: gnn, rpaulo
Reported by: gnn

Respect the libdir option when linking drti.o, rather than hardcoding the
default path of /usr/lib(32)/dtrace.

MFC after: 3 weeks

Remove drti.o's dependency on libelf. This makes it possible to add DTrace
probes to userland programs and libraries without also needing to link
libelf.

dtrace -G places the __SUNW_dof symbol at the beginning of the DOF (DTrace
probe and provider metdata) section in the generated object file; drti.o
now just uses this symbol to locate the section. A complication occurs
when multiple dtrace-generated object files are linked together, since the
__SUNW_dof symbol defined in each file is global. This is handled by
using objcopy(1) to convert __SUNW_dof to a local symbol once drti.o has
been linked with the generated object file. Upstream, this is done using a
linker feature not present in GNU ld.

Differential Revision: https://reviews.freebsd.org/D1757
Reviewed by: rpaulo
MFC after: 1 month
Relnotes: yes

Mechanically convert cddl sun #ifdef's to illumos

Since the upstream for cddl code is now illumos not sun, mechanically
convert all sun #ifdef's to illumos #ifdef's which have been used in all
newer code for some time.

Also do a manual pass to correct the use if #ifdef comments as per style(9)
as well as few uses of #if defined(__FreeBSD__) vs #ifndef illumos.

MFC after: 1 month
Sponsored by: Multiplay

Make DTrace stuff compile with C99 standard.

Revert r267898.

Bring the following change from the illumos-joyent repository:

commit 78e24ab6803bbe11ba37642624e1498ede5b239d
Author: Bryan Cantrill <bryan@joyent.com>
Date: Thu Oct 31 01:20:54 2013

OS-1688 DTrace count() with histogram
OS-2360 DTrace full width distribution histograms
OS-2361 DTrace frequency trails

MFC after: 2 weeks

Re-apply r248644. This fixes an annoying problem which caused dtrace -c to
fail to attach to stripped binaries. With the _r_debug_postinit symbol,
dtrace(1) can now set a breakpoint in the victim process after it has
registered its DOF table(s) with the kernel. r_debug_state cannot be used
for this purpose since it is called before DOF is made available, in which
case dtrace(1) cannot create USDT probes before the program begins
execution.

MFC after: 2 weeks

The uaddr, ufunc, umod and usym functions all seem to work as expected on
FreeBSD, so stop hiding them behind a "#if defined(sun)".

Reported by: Prashanth Kumar <pra_udupi@yahoo.co.in>

Add a function, memstr, which can be used to convert a buffer of
null-separated strings to a single string. This can be used to print the
full arguments of a process using execsnoop (from the DTrace toolkit) or
with the following one-liner:

dtrace -n 'syscall::execve:return {trace(curpsinfo->pr_psargs);}'

Note that this relies on the process arguments being cached via the struct
proc, which means that it will not work for argvs longer than
kern.ps_arg_cache_limit. However, the following rather non-portable
script can be used to extract any argv at exec time:

fbt::kern_execve:entry
{
printf("%s", memstr(args[1]->begin_argv, ' ',
args[1]->begin_envv - args[1]->begin_argv));
}

The debug.dtrace.memstr_max sysctl limits the maximum argument size to
memstr(). Thanks to Brendan Gregg for helpful comments on freebsd-dtrace.

Tested by: Fabian Keil (earlier version)
MFC after: 2 weeks

Load the dtraceall module if /dev/dtrace/dtrace doesn't exist.

MFC after: 3 days

revert r248644 because of the regression for usdt probes

USDT probes are advertised to kernel by initialization code with
atrribute((constructor))). It seems that on Solaris the .init-ish code
of the main object is executed before RD_PREINIT point is hit. On
FreeBSD that is not the case. And because on FreeBSD there is no other
well-defined point between RD_PREINIT and main() we have to parse a
DTrace script when main is hit, for time being.

A footnote: currently we actually post RD_POSTINIT event, but that's a
bug because the event is triggered by hitting r_debug_state which
happens before any init code is executed.

Reported by: markj

DTrace: Revert r249367

The following change from illumos brought caused DTrace to
pause in an interactive environment:

3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This was not detected during testing because it doesn't
affect scripts.

We shouldn't be changing the environment, especially since the
LD_NOLAZYLOAD option doesn't apply to our (GNU) ld.
Unfortunately the change from upstream was made in such a way
that it is very difficult to separate this change from the
others so, at least for now, it's better to just revert
everything.

Reference:
https://www.illumos.org/issues/3026

Reported by: Navdeep Parhar and Mark Johnston

dtrace: ensure that we can always catch a process (e.g. when -c is used)

It is not guaranteed that a program has a symbol table entry for main
and thus that it would be possible to set a breakpoint on it.

Reviewed by: rpaulo
Discussed with: rpaulo
MFC after: 13 days

Bring llquantize support into Dtrace.

Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

Special thanks to Mark Peek for providing fix to an
assertion and to Fabian Keill for testing the port.

Illumos Revision: 13355:15b74a2a9a9d

Reference:
https://www.illumos/issues/905

Obtained from: Illumos
Tested by: Fabian Keill, mp
MFC after: 4 days

Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread. At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particukar, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by: Sandvine Inc.
MFC after: 1 week

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Give a chance to the target binary to run the ctors by waiting until it
has reached main(). This allows plockstat to work.

Sponsored by: The FreeBSD Foundation

Add libdtrace support for tracing userland programs.

Summary of changes:
* Implement a compatibility shim between Solaris libproc and our
libproc and remove several ifdefs because of this.
* Port the drti to FreeBSD.
* Implement the missing DOODAD sections
* Link with libproc and librtld_db
* Support for ustack, jstack and uregs (by sson@)
* Misc bugfixing

When writing the SUWN_dof section, we had to resort to building the ELF
file layout by "hand". This is the job of libelf, but our libelf doesn't
support this yet. When libelf is fixed, we can remove the code under
#ifdef BROKEN_LIBELF.

Sponsored by: The FreeBSD Foundation

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Merge latest DTrace changes from Perforce.

Approved by: jb

- Import the HEAD csup code which is the basis for the cvsmode work.

* Add FreeBSD action extensions.
* Get the kernel module file name rather than hard-coding it like on Solaris.
* Use FreeBSD's process library API.
* Handle FreeBSD's different lock types.
* Get the list of loaded providers via a syscall.