Revert "dtrace: make 'ring' and 'fill' policies imply 'noswitch' flag"

This reverts commit e92491d95ff3500e140eafa614e88ca84ffb0d26.

The general idea looked good to me. In particular, it allowed to save
some memory and avoid memory allocation failures when a large buffer
size was requested along with ring and fill policies.

But I didn't take into account that the second, supposedly unused
buffer, was actually used as the scratch buffer. The scratch buffer is
used as a temporary space for DTrace subroutines like copyin, copyinstr,
and alloca.

I think that the change can be fixed by allocating a separate smaller
buffer for the scratch buffer, but that fix would require more work than
I am able to do now. Hence the revert.

Reported by: Domagoj Stolfa
Diagnosed by: Domagoj Stolfa, markj
MFC after: immediately

dtrace: make 'ring' and 'fill' policies imply 'noswitch' flag

This should disable allocation of the second per-CPU principal buffer
which is never used. This will also enable additional asserts
for buffers that are never switched.

dtrace: Fix a kernel panic in printm()

When using printm(), one should always pass a scratch pointer to it.
This is achieved by calling printm with memref

BEGIN { printm(fixed_len, memref(ptr, var_len)); }

which will return a pointer to the DTrace scratch space of size
sizeof(uintptr_t) * 2. However, one can easily call printm() as follows

BEGIN { printm(10, (void *)NULL); }

and panic the kernel as a result. This commit does two things:

(1) adds a new macro DTRACE_INSCRATCHPTR(mstate, ptr, howmany) which
checks if a certain pointer is in the DTrace scratch space;
(2) uses DTRACE_INSCRATCHPTR() to implement a check on printm()'s DIFO
return value in order to avoid the panic and sets CPU_DTRACE_BADADDR
if the address is not in the scratch space.

Reviewed by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D41722

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

dtrace: Improve a verifier error message

Using '%r0' in efunc causes it to parse %r as a 'r' specifier.

This diff just adds a '%' in front of '%r0' in order to create the
correct output.

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D38176

dtrace: Add a "regs" variable

This allows invop-based providers (i.e., fbt and kinst) to expose the
register file of the CPU at the point where the probe fired. It does
not work for SDT providers because their probes are implemented as plain
function calls and so don't save registers. It's not clear what
semantics "regs" should have for them anyway.

This is akin to "uregs", which nominally provides access to the
userspace registers. In fact, DIF already had a DIF_VAR_REGS variable
defined, it was simply unimplemented.

Usage example: print the contents of %rdi upon each call to
amd64_syscall():

fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}

Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.
Currently there are no similar definitions for non-x86 platforms.

Reviewed by: christos
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D36799

Adjust dtrace_getf_barrier() definition to avoid clang 15 warning

With clang 15, the following -Werror warnings is produced:

sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:17019:20: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
dtrace_getf_barrier()
^
void

This is because dtrace_getf_barrier() is declared with a (void) argument
list, but defined with an empty argument list. Make the definition match
the declaration.

MFC after: 3 days

Use KERNEL_PANICKED() in more places

This is slightly more optimized than checking panicstr directly. For
most of these instances performance doesn't matter, but let's make
KERNEL_PANICKED() the common idiom.

Reviewed by: mjg
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D35373

fd: rename fget*_locked to fget*_noref

This gets rid of the error prone naming where fget_unlocked returns with
a ref held, while fget_locked requires a lock but provides nothing in
terms of making sure the file lives past unlock.

No functional changes.

dtrace: add a knob to control maximum size of principal buffers

We had a hardcoded limit of 1/128-th of physical memory that was further
subdivided between all CPUs as principal buffers are allocated on the
per-CPU basis. Actually, the buffers could use up 1/64-th of the
memmory because with the default switch policy there are two buffers per
CPU.

This commit allows to change that limit.

Note that the discussed limit is per dtrace command invocation.
The idea is to limit the size of a single malloc(9) call, not the total
memory size used by DTrace buffers.

Reviewed by: markj
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D33648

dtrace: Disable getf() as it is broken on FreeBSD

getf() on FreeBSD calls _sx_slock(), _sx_sunlock() and fget_locked().
Furthermore, it does not set the per-core fault flag, meaning it
usually ends up in a double fault panic once getf() does get called,
especially from fbt.

Reviewing the DTrace Toolkit + a number of other scripts scattered
around FreeBSD, I have not been able to find one use of getf(). Given
how broken the implementation currently is, we disable it until it
can be implemented properly.

Also comment out a test in aggs/tst.subr.d for getf().

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33378

sys/cddl: remove extraneous semicolons

Fixes: 5a1b490d502e ("FreeBSD changes to vendor source.")
Fixes: 91eaf3e1831d ("Custom DTrace kernel module...")
MFC after: 1 week
Sponsored by: The FreeBSD Foundation

dtrace: use %zu format specifier for data of size_t type

Sponsored by: The FreeBSD Foundation

Teach DTrace that unaligned accesses are OK on aarch64, not just x86.

MFC after: 3 days
Reviewed: andrew
Differential Revision: https://reviews.freebsd.org/D29369

dtrace: Fix /"string" == NULL/ comparisons using an uninitialized value.

A test of this is funcs/tst.strtok.d which has this filter:

BEGIN
/(this->field = strtok(this->str, ",")) == NULL/
{
exit(1);
}
The test will randomly fail with exit status of 1 indicating that this->field
was NULL even though printing it out shows it is not.

This is compiled to the DTrace instruction set:
// Pushed arguments not shown here
// call strtok() and set result into %r1
07: 2f001f01 call DIF_SUBR(31), %r1 ! strtok
// set thread local scalar this->field from %r1
08: 39050101 stls %r1, DT_VAR(1281) ! DT_VAR(1281) = "field"
// Prepare for the == comparison
// Set right side of %r2 to NULL
09: 25000102 setx DT_INTEGER[1], %r2 ! 0x0
// string compare %r1 (strtok result) to %r2
10: 27010200 scmp %r1, %r2

In this case only %r1 is loaded with a string limit set to lim1. %r2 being
NULL does not get loaded and does not set lim2. Then we call dtrace_strncmp()
with MIN(lim1, lim2) resulting in passing 0 and comparing neither side.
dtrace_strncmp() handles this case fine and it already has been while
being lucky with what lim2 was [un]initialized as.

Reviewed by: markj, Don Morris <dgmorris AT earthlink.net>
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27671

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

Fix "current" variable name conflict with openzfs

The variable "current" is an alias for curthread
in openzfs. Rename all variable uses of current
in dtrace.c to curstate.

Replace read_random(9) with more appropriate arc4rand(9) KPIs

Reviewed by: ae, delphij
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D19760

Correct a comment.

Submitted by: Domagoj Stolfa
X-MFC with: r332364
Sponsored by: DARPA, AFRL

Assert that dtrace_probe() doesn't re-enter itself.

This helps catch cases where an instrumented function is called while
in probe context.

Submitted by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
MFC after: 2 weeks
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D14863

Fix an access of an uninitialized variable in dtrace_probe().

Reported by: Coverity, via cem
MFC after: 3 days

Use the thread's ucred struct when fetching jid or jailname.

Reported by: mjg
X-MFC with: r327888

Add "jid" and "jailname" variables to DTrace.

These return the jail ID and jail name for the traced process,
respectively, and are analogous to "zonename" on Solaris/illumos.
"zonename" is now aliased to "jailname".

Also add some stress tests for the new variables.

Submitted by: Domagoj Stolfa <domagoj.stolfa@gmail.com>
Reviewed by: dteske (previous version)
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13877

Use C99 initializers for DTrace provider methods.

This makes the definitions easier to read and more cscope-friendly.

MFC after: 1 week

o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by: ngie
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D11901

Ensure that the DOF string length is divisible by 2.

It is an ASCII encoding of a hexadecimal representation of the DOF file
used to enable anonymous tracing, so its length should always be even.

MFC after: 1 week

Use PC-relative relocations for USDT probe sites on i386 and amd64.

When recording probe site addresses in the output DOF file, dtrace -G
needs to emit relocations for the .SUNW_dof section in order to obtain
the addresses of functions containing probe sites. DTrace expects the
addresses to be relative to the base address of the final ELF file,
and the amd64 USDT implementation was relying on some unspecified and
incorrect behaviour in the base system GNU ld to achieve this.

This change reimplements the probe site relocation handling to allow
USDT to be used with lld and newer GNU binutils. Specifically, it
makes use of R_X86_64_PC64/R_386_PC32 relocations to obtain the
probe site address relative to the DOF file address, and adds and uses a
new DOF relocation type which computes the final probe site address using
these relative offsets.

Reported by and discussed with: Rafael Espíndola
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D9374

Replace the implementation of DTrace's RAND subroutine for generating
low-quality random numbers with a modern implementation (xoroshiro128+)
that is capable of generating better quality randomness without compromising performance.

Submitted by: Graeme Jenkinson
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D9051

Remove the DTRACEHIOC_ADD ioctl.

This ioctl has been considered legacy by upstream since the DTrace code
was first imported, and is unused. The removal also allows some
simplification of dtrace_helper_slurp().

Also remove a bogus copyout in the DTRACEHIOC_ADDDOF handler. Due to a
bug, it would overwrite an in-memory copy of the DOF header rather than
the passed-in DOF helper. Moreover, DTRACEHIOC_ADDDOF already copies the
helper back out automatically since its argument has the IOC_OUT attribute.

Have DTrace handle faults when dereferencing a lock object pointer.

MFC after: 1 week

Ignore LC_SLEEPABLE when testing whether a mutex is adaptive.

MFC after: 1 week

Fix a kernel panic in DTrace's rw_iswriter subroutine.
On FreeBSD the sense of rw_write_held() and rw_iswriter() were reversed,
probably due to a cut and paste error. Using rw_iswriter() would cause
the kernel to panic.

Reviewed by: markj
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D8718

Add tunable to disable destructive dtrace

Submitted by: Joerg Pernfuss <code.jpe@gmail.com>
Reviewed by: rstone, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D8624

Remove the DTrace printt and typeref actions.

These are FreeBSD-specific and were added in r178576 to provide the ability
to pretty-print instances of compound types. However, the print action has
long since been augmented to provide this functionality with a simpler
interface.

Discussed with: gnn
Differential Revision: https://reviews.freebsd.org/D8478

Avoid truncating the return value of DTrace predicates.

Predicates are DIF objects whose return value is compared with zero to
determine whether the corresponding probe body is to be executed. The return
value itself is the contents of a 64-bit DIF register, but it was being
truncated to an int before the comparison. This meant that a predicate such
as /0x100000000/ would evaluate to false.

Reported by: rwatson
MFC after: 3 days

Add initial DTrace support for RISC-V.

Sponsored by: DARPA, AFRL
Sponsored by: HEIF5

Allow DOF sections with excessively long probe function components.

Without this change, DTrace will refuse to load a DOF section if the
function component of any of its probes exceeds DTRACE_FUNCNAMELEN (128).
Probes in C++ programs can have very long function components. Rather than
rejecting all probes if a single probe exceeds the limit, simply skip the
invalid probe and emit a warning. This ensures that valid probes are
instantiated.

PR: 207735
MFC after: 2 weeks

Implement support for boot-time DTrace.

This allows one to enable DTrace probes relatively early during boot,
during SI_SUB_DTRACE_ANON, before dtrace(1) can invoked. The desired
enabling is created using dtrace -A, which writes a /boot/dtrace.dof
file and uses nextboot(8) to ensure that DTrace kernel modules are loaded
and that the DOF file describing the enabling is loaded by loader(8)
during the subsequent boot. The trace output can then be fetched with
dtrace -a.

With this commit, boot-time DTrace is only functional on i386 and amd64: on
other architectures, the high-resolution timer frequency is initialized
during SI_SUB_CLOCKS and is thus not available when the anonymous
tracing state is initialized. On x86, the TSC is used and is thus available
earlier.

MFC after: 1 month
Relnotes: yes

Remove unused variables dtrace_in_probe and dtrace_in_probe_addr.

Modify DTRACEHIOC_ADDDOF to copy the DOF section from the target process.

r281257 added support for lazyload mode by allowing dtrace(1) to register
a DOF section on behalf of a traced process. This was implemented by
having libdtrace copy the DOF section into a heap-allocated buffer and
passing its address to the ioctl handler. However, DTrace uses the DOF
section address as a lookup key in certain cases, so the ioctl handler
should be given the target process' DOF section address instead. This
change modifies the ADDDOF handler to copy the DOF section in from the
target process, rather than from dtrace(1).

Avoid dereferencing curthread->td_proc->p_cred in DTrace probe context.

When a process is exiting, there is a narrow window where p_cred may be
NULL while its threads are still executing. Specifically, the last thread
to exit a process sets the process state to PRS_ZOMBIE with the proc
spinlock held and then calls thread_exit(). thread_exit() drops the spin
lock, permitting the process to be reaped and thus causing its cred struct
to be released. However, the exiting thread may still cause DTrace probes
to fire by calling sched_throw(), resulting in a double fault if such a
probe enabling attempts to access the GID or UID DIF variables.

The thread's cred reference is not susceptible to this race since it is not
released until after the thread has exited.

MFC after: 1 week

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten
years for head. However, it is continuously misused as the mpsafe argument
for callout_init(9). Deprecate the flag and clean up callout_init() calls
to make them more consistent.

Differential Revision: https://reviews.freebsd.org/D2613
Reviewed by: jhb
MFC after: 2 weeks

Fix DTrace's panic() action.

It would previously call into some unfinished Solaris compatibility code and
return without actually calling panic(9). The compatibility code is
unneeded, however, so just remove it and have dtrace_panic() call vpanic(9)
directly.

Differential Revision: https://reviews.freebsd.org/D2349
Reviewed by: avg
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

libdtrace: add support for lazyload mode.

Passing "-x lazyload" to dtrace -G during compilation causes dtrace(1) to
not link drti.o into the output object file, so the USDT probes are not created
during process startup. Instead, dtrace(1) will automatically discover and
create probes on the process' behalf when attaching.

Differential Revision: https://reviews.freebsd.org/D2203
Reviewed by: rpaulo
MFC after: 1 month

Initial version of DTrace on ARM32.

Submitted by: Howard Su based on work by Oleksandr Tymoshenko
Reviewed by: ian, andrew, rpaulo, markj

Fix a typo in r278137: make sure to free provider state.

X-MFC-With: r278136

Continue to handle the case where state is NULL, though this currently
cannot happen on FreeBSD. r278136 overlooked the fact that a destructor
registered with devfs_set_cdevpriv(9) is invoked even in the case of an
error.

X-MFC-With: r278136

Diff reduction with illumos, in preparation for merging r266993 from the
vendor branch. No functional change.

MFC after: 1 week

Mechanically convert cddl sun #ifdef's to illumos

Since the upstream for cddl code is now illumos not sun, mechanically
convert all sun #ifdef's to illumos #ifdef's which have been used in all
newer code for some time.

Also do a manual pass to correct the use if #ifdef comments as per style(9)
as well as few uses of #if defined(__FreeBSD__) vs #ifndef illumos.

MFC after: 1 month
Sponsored by: Multiplay

remove opensolaris cyclic code, replace with high-precision callouts

In the old days callout(9) had 1 tick precision and that was inadequate
for some uses, e.g. DTrace profile module, so we had to emulate cyclic
API and behavior. Now we can directly use callout(9) in the very few
places where cyclic was used.

Differential Revision: https://reviews.freebsd.org/D1161
Reviewed by: gnn, jhb, markj
MFC after: 2 weeks

Follow up to r225617. In order to maximize the re-usability of kernel code
in userland rename in-kernel getenv()/setenv() to kern_setenv()/kern_getenv().
This fixes a namespace collision with libc symbols.

Submitted by: kmacy
Tested by: make universe

Return 0 for the PPID of threads in process 0, as process 0 doesn't have a
parent process.

MFC after: 2 weeks

Merge from OpenSolaris (24-Jul-2010):

6679140 asymmetric alloc/dealloc activity can induce dynamic variable drops
6679193 dtrace_dynvar walker produces flood of dtrace_dynhash_sink

This finishes a set of merges from the older OpenSolaris releases.
Still the FreeBSD port has many differences that are difficult to
account for but that seems normal given that the kernels are different.

MFC after: 1 week

Merge from OpenSolaris (30-Jun-2009):

6851093 system drops to kmdb with anonymous dtrace probes + kmdb

This has no effect on FreeBSD (code is ifdef'ed) but is useful as
reference for future merges.

MFC after: 1 week

Merge from OpenSolaris (22-Apr-2008):

6823388 DTrace ioctl handlers must validate all structure members

MFC after: 1 week

Merge from OpenSolaris (20-Apr-2008):

6822482 DOF validation needs to handle loadable sections flagged as unloadable

MFC after: 1 week

Merge from OpenSolaris (15-Sep-2008):

6735480 race between probe enabling and provider registration

MFC after: 1 week

Small merges from OpenSolaris:

These have no effect on FreeBSD, in fact they are ifdef'ed,
but make easier future merges:

6699767 panic in spec_open()

6718877 crgetzoneid() use can cause problems when forking processes with
USDT providers in a non global zone

MFC after: 3 days

Continue the crusade towards a dev_clone()-free kernel, removing its
usage from dtrace. The dtrace code already uses cdevpriv(9) since FreeBSD
8, so this change should be quite harmless.

Reviewed by: markj
Approved by: markj
MFC after: never

MFV r258379;

4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision: 4a20ab41aadcb81c53e72fc65886e964e9add59

Reference:
https://www.illumos.org/issues/4248
https://www.illumos.org/issues/4249

Obtained from: Illumos
MFC after: 1 month

Expose a few DTrace parameters as sysctls under kern.dtrace and add
descriptions for several existing sysctls.

PR: 187027
Submitted by: Fedor Indutny <fedor@indutny.com> (original version)
MFC after: 2 weeks

1452 DTrace buffer autoscaling should be less violent

illumos/illumos-gate@6fb4854bed54ce82bd8610896b64ddebcd4af706

This fixes the tst.resize1.d and tst.resize2.d DTrace tests, which have
been failing since r261122 since they were causing dtrace(1) to attempt to
allocate and use large amounts of memory, and get killed by the OOM killer
as a result.

MFC after: 1 month

Revert r260091. The vmem calls seem to be slower than the *_unr() calls that
they replaced, which is important considering that probe IDs are allocated
during process startup for USDT probes.

Now that vmem(9) is available, use vmem arenas to allocate probe and
aggregation IDs, as is done in the upstream illumos code. This still
requires some FreeBSD-specific code, as our vmem API is not identical to the
one in illumos.

Submitted by: Mike Ma <mikemandarine@gmail.com>

Add a function, memstr, which can be used to convert a buffer of
null-separated strings to a single string. This can be used to print the
full arguments of a process using execsnoop (from the DTrace toolkit) or
with the following one-liner:

dtrace -n 'syscall::execve:return {trace(curpsinfo->pr_psargs);}'

Note that this relies on the process arguments being cached via the struct
proc, which means that it will not work for argvs longer than
kern.ps_arg_cache_limit. However, the following rather non-portable
script can be used to extract any argv at exec time:

fbt::kern_execve:entry
{
printf("%s", memstr(args[1]->begin_argv, ' ',
args[1]->begin_envv - args[1]->begin_argv));
}

The debug.dtrace.memstr_max sysctl limits the maximum argument size to
memstr(). Thanks to Brendan Gregg for helpful comments on freebsd-dtrace.

Tested by: Fabian Keil (earlier version)
MFC after: 2 weeks

Initialize and free the DTrace taskqueue in the dtrace module load/unload
handlers rather than in the dtrace device open/close methods. The current
approach can cause a panic if the device is closed which the taskqueue
thread is active, or if a kernel module containing a provider is unloaded
while retained enablings are present and the dtrace device isn't opened.

Submitted by: gibbs (original version)
Reviewed by: gibbs
Approved by: re (glebius)
MFC after: 2 weeks

Rename the kld_unload event handler to kld_unload_try, and add a new
kld_unload event handler which gets invoked after a linker file has been
successfully unloaded. The kld_unload and kld_load event handlers are now
invoked with the shared linker lock held, while kld_unload_try is invoked
with the lock exclusively held.

Convert hwpmc(4) to use these event handlers instead of having
kern_kldload() and kern_kldunload() invoke hwpmc(4) hooks whenever files are
loaded or unloaded. This has no functional effect, but simplifes the linker
code somewhat.

Reviewed by: jhb

Use kld_{load,unload} instead of mod_{load,unload} for the linker file load
and unload event handlers added in r254266.

Reported by: jhb
X-MFC with: r254266

FreeBSD's DTrace implementation has a few problems with respect to handling
probes declared in a kernel module when that module is unloaded. In
particular,

* Unloading a module with active SDT probes will cause a panic. [1]
* A module's (FBT/SDT) probes aren't destroyed when the module is unloaded;
trying to use them after the fact will generally cause a panic.

This change fixes both problems by porting the DTrace module load/unload
handlers from illumos and registering them with the corresponding
EVENTHANDLER(9) handlers. This allows the DTrace framework to destroy all
probes defined in a module when that module is unloaded, and to prevent a
module unload from proceeding if some of its probes are active. The latter
problem has already been fixed for FBT probes by checking lf->nenabled in
kern_kldunload(), but moving the check into the DTrace framework generalizes
it to all kernel providers and also fixes a race in the current
implementation (since a probe may be activated between the check and the
call to linker_file_unload()).

Additionally, the SDT implementation has been reworked to define SDT
providers/probes/argtypes in linker sets rather than using SYSINIT/SYSUNINIT
to create and destroy SDT probes when a module is loaded or unloaded. This
simplifies things quite a bit since it means that pretty much all of the SDT
code can live in sdt.ko, and since it becomes easier to integrate SDT with
the DTrace framework. Furthermore, this allows FreeBSD to be quite flexible
in that SDT providers spanning multiple modules can be created on the fly
when a module is loaded; at the moment it looks like illumos' SDT
implementation requires all SDT probes to be statically defined in a single
kernel table.

PR: 166927, 166926, 166928
Reported by: davide [1]
Reviewed by: avg, trociny (earlier version)
MFC after: 1 month

Hide references to mod_lock. In FreeBSD it is always acquired with the
provider lock held, so its use has no effect.

DTrace: Revert r249367

The following change from illumos brought caused DTrace to
pause in an interactive environment:

3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This was not detected during testing because it doesn't
affect scripts.

We shouldn't be changing the environment, especially since the
LD_NOLAZYLOAD option doesn't apply to our (GNU) ld.
Unfortunately the change from upstream was made in such a way
that it is very difficult to separate this change from the
others so, at least for now, it's better to just revert
everything.

Reference:
https://www.illumos.org/issues/3026

Reported by: Navdeep Parhar and Mark Johnston

Dtrace: dtrace.c erroneously checks for memory alignment on amd64.

Merge change from illumos:

3511 dtrace.c erroneously checks for memory alignment on amd64

Illumos Revision: c93cc65

Reference:
https://www.illumos.org/issues/3511

Obtained from: Illumos
MFC after: 3 weeks

Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.

There is one known issue: Some probes will display an error message along the
lines of: "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit. I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded. Volunteers are welcome.

MFC after: 1 month

Use a proper destructor function.

When calling a revoke(2) on a dtrace device, dtrace_close() could be
called, even if threads are still stuck in the device. Defer the actual
deallocation of datastructures to the cdevpriv destructor.

While there, remove the unneeded D_TRACKCLOSE and D_NEEDMINOR flags. For
the helper device, we never need it. For the regular dtrace devices, we
only need these flags on FreeBSD pre-8.

MFC after: 1 month

Streamline use of cdevpriv and correct some corner cases.

1) It is not useful to call "devfs_clear_cdevpriv()" from
"d_close" callbacks, hence for example read, write, ioctl and
so on might be sleeping at the time of "d_close" being called
and then then freed private data can still be accessed.
Examples: dtrace, linux_compat, ksyms (all fixed by this patch)

2) In sys/dev/drm* there are some cases in which memory will
be freed twice, if open fails, first by code in the open
routine, secondly by the cdevpriv destructor. Move registration
of the cdevpriv to the end of the drm open routines.

3) devfs_clear_cdevpriv() is not called if the "d_open" callback
registered cdevpriv data and the "d_open" callback function
returned an error. Fix this.

Discussed with: phk
MFC after: 2 weeks

Bump dtrace_helper_actions_max from 32 to 128

Dave Pacheco from Joyent (and Dtrace.org) bumped the cap to 1024 but,
according to his blog, 128 is the recommended minimum.

For now bump it safely to 128 although we may have to bump it further
if there is demand in the future.

Reference:

http://www.illumos.org/issues/2558
http://dtrace.org/blogs/dap/2012/01/50/where-does-your-node-program-spend-its-time/

Bring llquantize support into Dtrace.

Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

Special thanks to Mark Peek for providing fix to an
assertion and to Fabian Keill for testing the port.

Illumos Revision: 13355:15b74a2a9a9d

Reference:
https://www.illumos/issues/905

Obtained from: Illumos
Tested by: Fabian Keill, mp
MFC after: 4 days

Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread. At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particukar, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by: Sandvine Inc.
MFC after: 1 week

Add MIPS support to cddl/contrib part:

- header and stub .c file for fasttrap module. It's not supported on
MIPS yet, but there is no way to disable support completely
- Do as amd64 trying to limit allocated memory

Revert the approach for skipping lockstat_probe_func call when doing
lock_success/lock_failure, introduced in r228424, by directly skipping
in dtrace_probe.

This mainly helps in avoiding namespace pollution and thus lockstat.h
dependency by systm.h.

As an added bonus, this also helps in MFC case.
Reviewed by: avg
MFC after: 3 months (or never)
X-MFC: r228424

In order to maximize the re-usability of kernel code in user space this
patch modifies makesyscalls.sh to prefix all of the non-compatibility
calls (e.g. not linux_, freebsd32_) with sys_ and updates the kernel
entry points and all places in the code that use them. It also
fixes an additional name space collision between the kernel function
psignal and the libc function of the same name by renaming the kernel
psignal kern_psignal(). By introducing this change now we will ease future
MFCs that change syscalls.

Reviewed by: rwatson
Approved by: re (bz)

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Fix two bugs in DTrace:
* when the process exits, remove the associated USDT probes
* when the process forks, duplicate the USDT probes.

Sponsored by: The FreeBSD Foundation

Kernel DTrace support for:
o uregs (sson@)
o ustack (sson@)
o /dev/dtrace/helper device (needed for USDT probes)

The work done by me was:
Sponsored by: The FreeBSD Foundation

Update several places that iterate over CPUs to use CPU_FOREACH().

Add the OpenSolaris dtrace lockstat provider. The lockstat provider
adds probes for mutexes, reader/writer and shared/exclusive locks to
gather contention statistics and other locking information for
dtrace scripts, the lockstat(1M) command and other potential
consumers.

Reviewed by: attilio jhb jb
Approved by: gnn (mentor)

Merge latest DTrace changes from Perforce.

- Import the HEAD csup code which is the basis for the cvsmode work.

Disable use of the user credentials until there is code to set the levels
that DTrace uses.

This fixes a bug that would have affected kernels built with MAC and all
kernels built after the mpsafetty integration.

The bug will be apparent in RELENG7 on MAC kernels.

Reported by: kan

Replace all calls to minor() with dev2unit().

After I removed all the unit2minor()/minor2unit() calls from the kernel
yesterday, I realised calling minor() everywhere is quite confusing.
Character devices now only have the ability to store a unit number, not
a minor number. Remove the confusion by using dev2unit() everywhere.

This commit could also be considered as a bug fix. A lot of drivers call
minor(), while they should actually be calling dev2unit(). In -CURRENT
this isn't a problem, but it turns out we never had any problem reports
related to that issue in the past. I suspect not many people connect
more than 256 pieces of the same hardware.

Reviewed by: kib

Add calls to callout_drain() to ensure the callouts are flushed before
we free memory from underneath them.

This fixes an occasional panic I've been seeing in softclock() where a bad
pointer would be encountered when pushing DTrace hard.

Don't enforce unique device minor number policy anymore.

Except for the case where we use the cloner library (clone_create() and
friends), there is no reason to enforce a unique device minor number
policy. There are various drivers in the source tree that allocate unr
pools and such to provide minor numbers, without using them themselves.

Because we still need to support unique device minor numbers for the
cloner library, introduce a new flag called D_NEEDMINOR. All cdevsw's
that are used in combination with the cloner library should be marked
with this flag to make the cloning work.

This means drivers can now freely use si_drv0 to store their own flags
and state, making it effectively the same as si_drv1 and si_drv2. We
still keep the minor() and dev2unit() routines around to make drivers
happy.

The NTFS code also used the minor number in its hash table. We should
not do this anymore. If the si_drv0 field would be changed, it would no
longer end up in the same list.

Approved by: philip (mentor)

Merge a recent change from the OpenSolaris source tree.
(Don't ask for a vendor import of this yet, we're in the early days of svn)

Instead of using cyclic timers to call the state clean and deadman callbacks,
use a callout on FreeBSD to avoid the deadlock on FreeBSD due to trying to
send interprocessor interrupts with interrupts disabled.

Reported by: ps, jhb, peter, thompsa

Comment out the code that breaks with invariants. This is stuff that is
still WIP along with the lockstat provider, so there is no harm leaving
it out for now.

FreeBSD changes to vendor source.